Load Testing is an integral part of the performance testing function. Companies have long understood the importance of it and have been giving it the due attention in the overall software testing ecosystem. If so, why do we continue to see issues around handling peak loads? Flipkart, one of the largest Indian online retailers, failed to meet the load that came in on the Big Billion Day sale recently and had to extend an open apology to its customers in its lack of ability to do so. While it is not only a lost business the day such a failure happens, lack of load handling capabilities, result in lost future business on such peak days, more than any other form of quality. What this incident tells me as an individual in quality assurance is three things –
a. Understand your data as you prepare to test – Collect data on business metrics you may have, the load you anticipate, any past numbers you may have (from previous such peak days), what kind of issues you have faced in the past, specific patterns such as day/time/region of load – All of this data is valuable in helping you take into account your current release’s load test effort, rather than carrying on age old performance tests that run release after release
b. Repeat your test a few days before your D Day – Even if you have done a detailed performance test say a few months back and you have an upcoming BIG day such as the one above for Flipkart or the upcoming Cyber Monday post the Thanksgiving season in the US, it definitely is worth to run a repeat test a few days before your D day to ensure you have all your ducks lined up
c. Take the moment to anticipate a customer’s pain individually – For you as an organization that handles a large load, you may miss looking at each customer individually, when you think of the big picture. At times, it definitely helps to push the big picture into the back burner a bit and take the time to understand the frustration an individual customer would face by the lack of your application’s availability on a day that they have planned to shop with you. For them the load you have does not matter. The availability or lack thereof, of your application is what really matters. So flip the coin on their side and take the time to understand individual pains, and help your team also understand this to take the human element into account while planning the load testing effort. This will encourage them to do some additional homework such as what loads are expected in the current shopping season, what competitors are offering, any additional tools to leverage etc. to tie these back to the metrics you have in mind.
Together these above points, will help bring objectivity with a human touch in your load testing effort, giving an uplift to your overall performance testing effort.