While the core scenarios to test, covering all paths through the engineered code is very important in a quality engineering effort, equally important is the depth and quality of test data being used. Poor or inadequate test data often leave even the best of the test cases incomplete. It is all the more reason today for test data to be comprehensive given how test efforts are often not fully scripted and a lot of exploratory testing is taking up. The quality of such an effort depends not just on the creative scenarios that the tester tries but equally so on the positive, negative, null and border line test data inputs that are provided. Yet another reason for the test data to be robust is the amount of automation that is happening and is paramount today. With unattended runs, the test data, their state after a test execution, the eligibility for reuse, the validity and ethical element in leveraging the most representative (real life like) test data are all critical today than ever before.
Having talked about the importance of reliable test data, looking at the testing landscape, a lot more is happening today at the web services and system level with minimal focus on the UI level. Obviously, web services testing is enabling engineering teams catch issues at a modular level early on in the cycle, promoting devops and continuous testing in parallel. If we closely look at web services testing, it is not really rocket science. In fact it is so modular with clear parameterization that anyone who knows how functions/methods work, can easily grow into this space. However, what is becoming important even in web services testing is the quality of test data. Herein again, the tester is working his/her way into enhanced test coverage by visualizing possible use cases of the web services and what kinds of data they need to be validated with in the testing cycle. Relative to UI testing where a lot of importance is given to workflows and paths, the success of a web services test effort increasingly relies on the test data used.
Moving on, yes we understand the importance of test data – what next? Generation of test data is a huge task in itself. Even expert testers may not always get this solutioning right – if so what is the scenario with junior / inexperienced testers? Sensitizing the team on importance of test data and looking at clear samples go a long way. In addition, peer reviews, team brainstorming sessions, discussions with business teams and core engineering teams also help. Additionally this is where Artificial Intelligence and Machine Learning will help in the coming years. Machine Learning models can be built to continuously look at not just test data but also live user data – with a clear set of rules defined in building and training the models, automatically generating test data is not far from reality. It is already in use in some places today, but is largely still in an R&D mode. Soon, this will become mainstream, further enabling a solid test data backed quality engineering effort, especially at the web services layer, making DevOps an engineering paradigm and success.