Compliance testing especially in areas such as accessibility, security have been on the rise in recent years. Whether mandated by end user needs or governmental requirements, it is welcoming to see more awareness and acknowledgement for these non-functional test areas.
However most teams still approach these as a one off test effort where compliance checks are done on a standalone basis. This mind-set and implementation needs to change as compliance test efforts are also slowly falling into the main stream regression suite needs. For example a WCAG compliance testing done on a given build of a certain release may only help you bench mark where you standard on the accessibility implementation aspects. Subsequently as the product undergoes multiple revisions in a given release the original set of tests that were done will not be fully valid as is. This is where timing the non-functional test effort in a release window, automating core regression candidates, building a strong scenario suite based on WCAG testing guidelines will all help in creating a robust and repeatable strategy. For example, when we talk about timing, it does not make sense to take up an accessibility test effort at the start of the release. One would rather wait for all functional elements to be in place (like in case of load testing for web applications) before an accessibility effort can commence. That said, one can’t wait until later in the game to understand the picture. It is becoming increasingly important for compliance testers to be involved upfront in reviewing the product design and architecture, test the requirements to ensure they will address the mandates down the line (assuming the engineering implementation adheres to the specifications).
Such upfront review and planning, with close monitoring during implementation and a thorough validation post development, along with a focus on repeating core compliance scenarios through regression, go a long way in establishing a repeatable strategy in today’s agile world – and this is the nimbleness product teams have come to expect of quality teams too.