Why Real User Testing Is Inevitable In Accessibility

Role based testing is a valuable practice across all test disciplines. Areas of the product and associated quality attributes that have a strong user connect, have heavily relied on role based testing. Herein, the tester himself assumes the role of the end user, in addition to several user connect programs through usability studies, user acceptance testing, beta testing, MVP programs where a select group of users are involved in the SDLC amongst others. In recent times, testing with emotional quotient is picking steam in being able to truly understand end user requirements, and ensuring they have translated into the product to release.

Of the areas that truly benefit from user based testing, web accessibility testing is one that stands out. As part of the usability umbrella, the user experience quotient is very high in the space of accessibility. The one thing unique in accessibility testing, is that a tester herein can step into a user’s shoes only with a certain level of accurate representation. In our experience of extensive accessibility testing over the last decade helping several global ISVs ship products that are legally compliant and user experience rich for accessibility standards, we have found several interesting patterns and data. An accessibility testing strategy and effort taken up purely by SMEs gives about 40% test coverage; a test effort taken up by real users (be it visually challenged, hearing challenged etc.) gives about 60% coverage; whereas accessibility with paired testing approach, that brings in the SMEs and real users together at the right times, gives a solid near 100% coverage. Paired accessibility testing is a unique test approach at QA InfoTech that we have been offering to our clients and training them on, for several years now.

A few specific examples of test scenarios that are better handled by real users (for example non-sighted users) include: verifying unexpected focus changes in screen reader, order or structure in which screen reader reads out the content, performing timed activities – these are ones that are best done by a real user, and would be difficult for a trained SME to take up individually. This is also because the sighted users often make assumptions, whereas the non-sighted real users test based on practical real time application usage.

Similarly, a few that are best found through a paired testing approach, include: testing for hidden elements, testing for elements that may be skipped by the screen reader, correctness of alt-txt tag values where the combined power of the SME and the real user goes a long way in ensuring realistic web accessibility testing.

Today accessibility has come a long way, with organizations not looking at it purely from a legal compliance standpoint, given the global digitization numbers and rising population that needs inclusive solutions. Technology is helping build proactive solutions in minimizing global disability foot print – while that is welcoming, as responsible product development and services organizations, the best we can do is to embrace accessibility roping in real users to test, thereby not just making the product more compliant but also generating respectable career options for our differently abled friends.

About the Author

Rajini Padmanaban

Rajini Padmanaban

As Vice President, Testing Services and Engagements, Rajini Padmanaban leads the engagement management for some of QA InfoTech's largest and most strategic accounts. She has over seventeen years of professional experience, primarily in software quality assurance. Rajini advocates software quality through evangelistic activities including blogging on test trends, technologies and best practices. She is also a regular speaker in conferences run by SQE, QAI STC ,ATA, UNICOM, EuroStar and has orchestrated several webinars. Her writings are featured in TechWell, Sticky Minds, Better Software Magazine. She has co-authored a book on crowdsourced testing . She can be reached at rajini.padmanaban@qainfotech.com

Related Posts