performance – QA InfoTech

24 Dec, 2015

The pros and cons of relying on time to interact

Rajini Padmanaban
Blogs
Tags: interactive, optimization, performance
no comments

Performance has certainly become a critical variable in the non-functional evaluation of an application. An application of acceptable performance metrics, enhances the user experience thereby increasing user stickiness and brand loyalty. For these obvious reasons, engineering organizations have been increasingly focusing on ensuring the performance of the application in the recent times. I was at a conference last week, where one of the presenters talked in detail about various things we as testers could do to help improve application performance, in simple but powerful ways. One of the points he harped on was to focus on the “Time to Interact (TTI)” metric, where large e-commerce giants are focusing on first loading elements of the page that users would be most interested in. While their focus is now caught in such elements, the rest of the application can be loaded in the background without they noticing the same. This is certainly a smart approach of a minimum load solution thereby optimizing their overall efforts and at the same satisfying the user and meeting his experience expectations. In today’s competitive world such approaches definitely go a long way in giving oneself a differentiation and an advantage. If so, is this solution a silver bullet to solve an application performance challenges, not posing any issues at all? This is the question that organizations now need to answer to reap the most of what TTI has to offer. I see two main challenges that need to be addressed out here:

Firstly, in their initial target of getting a good TTI, organizations can become overly focused just on this number. They may lose sight of the larger goal of the load time of the remaining components, which the users will soon start looking out for. Let’s say, I go to a shopping page and I know exactly what I want. I load it into my cart and head straight to checkout. Here as a user, my connect time with the TTI elements on the page is very low. The application should account for such scenarios too and keep in mind that while TTI focus is good, within the following few milliseconds to seconds, the rest of the elements on the page should also fully load – both these should be adequately accounted for in the performance testing effort and exit criteria should be defined accordingly.

Secondly, understanding what interactive elements users need and prioritizing to load them first is a humongous task of its own. TTI as a metric is easier said than done, as users are turning out to be global today – they come from various demographic backgrounds and their user personas are very different. With such a global base, what is relevant to one may not be fully relevant to another. Thus TTI has to be backed with adequate data analytics in making the informed call on which elements matter the most to the largest subsection of the user base.

Overall, while TTI is a good metric to leverage in the performance of an application, over-reliance on it can bring in detrimental performance which is more visible than in a scenario where TTI optimization has not been done. As long as engineering organizations understand this and account for it, in their development and testing efforts, TTI certainly yields a net positive impact.

09 Sep, 2015

Can functional testers become performance testers

Performance Testing has become one of the core niche areas for a tester to specialize in, in the non-functional testing space over the last decade. Performance has become a core differentiating factor in enabling a product to succeed in the marketplace. End users low tolerance for poor performance has impacted several products’ acceptance adversely in recent times. All of these along with an enhanced appreciation and understanding of how software performance can be implemented has given a new face lift for the performance testing and testers. The question is can functional testers be good performance testers.

At the very core, performance testing also leverages core functional scenarios, especially E2E scenarios and simulates them to be run under realistic and unrealistic load conditions that the business anticipates to see how the application performs. Benchmarking, profiling, tuning are some of the core activities performance testers take on, as part of their responsibilities. They also take on capacity planning to proactively plan for future performance expectations and competitive performance runs to understand how the application fares against other players in the market. From an implementation standpoint, performance testers take on a lot of automation with some amount of manual testing too, especially for the analysis and quick runs around page load and response times.

From all of these, it is not impossible for a functional tester to take on performance testing, but the real challenge or the area the tester needs to focus on changing is the mind-set. Functional testers come in with a very strong bias on functional workflows. They will have to move past this to think performance – while they leverage functional workflows for performance testing, the focus should be on load, scalability, rendering, response times and overall performance. This change does not happen overnight. If this can be inculcated along with a strong understanding of how these performance and functional scenarios maps to reaping business value for the product, there is no reason to stop a functional tester from specializing in the space of performance testing.