Promoting Repeatability in Defect Finding

Being able to reproduce defects is a very important step in defect management to aid quick defect analysis, fix and closure. Screen snapshots highlighting the defect, a video of the series of steps to see the defect have been traditionally used as extras, to help reproduce the defect in addition to the “Steps to Reproduce” that a tester outlines in his defect report. Whether the tester is working in a paired development mode, sitting beside his developer, or is co-located yet not in the same room as that of his developer or is remotely located from his development team (especially in cases where a testing services vendor may be hired), the importance of reproducing defects can never be underestimated. In a large sense, this also speaks for the reputation that tester has on his team. Testing in the current day has become very complex having to deal with varied parameters in its scope of operations – for example, what are the chances I will be able to consistently reproduce a defect filed by a tester in Africa who has found an issue on a specific version of his mobile device? Given the variables at play, it is now more important than ever before for a tester to take conscious steps to maintain the sanctity of “defect reproduction”. Hereare some steps to promote better defect repeatability in our test efforts:
1. Be cognizant of your test configuration even in an exploratory test effort

2. Try to bring in a sequence and logical flow in your test effort rather than trying haphazard and random steps. This will help you sequentially troubleshoot and determine your steps to reproduce the issue with improved chances of seeing the issue. This does not mean curbing your creative juices in exploring newer tests, but this is where the convergence of creative and critical thinking, plays an important role

3. Take time to analyse an issue once found from possible other areas that might be affected – for example, other locales, other devices, other operating systems which might face the same issue

4. Stay current with forums that discuss issues with specific devices, browsers, operating systems that help you differentiate product specific issues from external issues

5. Give time to reproduce certain issues. Some issues show up only with time gaps. Experts suggest taking breaks at times (say a coffee break) to come back and revisit an issue. Leave the system on, at a logical state then to enable it to process the information you have provided. For all you know when you come back you would see the issue reproduced. Such gaps also help you come back refreshed to look at the issue from a possibly different perspective

6. Discuss the issue with your team members/developers. For all you know, they may have views on the issue, that you have not thought about

7. Explore ways to give the developer access to your machine/environment to see the issue, wherever possible, instead of giving just snapshots and video clippings

8. If in doubt, take the time yourself to reproduce the issue a few times, at intervals before reporting the issue. For all you know the issue shows up only at specific times of the day influenced by users and their usage patterns at that point. Such extra details go a long way in building credibility for the defect and for yourself
While every attempt is made to reproduce a defect, sometimes it may just not be worth the effort, especially when it has been reported by a customer and it appears to be very time consuming to try to reproduce it. In such cases the team would be better off brainstorming what might have happened, where, why and proceed with analysing the code to see how to fix it. The tester in this case, plays an important role in seeing if any quick reproductions can be attempted; if not trying to see how to help with the debugging and troubleshooting with the goal of fixing the issue and regressing it. The initial analysis that was done to see if it can be reproduced, will help the tester at a later stage when the regression effort is taken on.
So, while the tester takes every sincere attempt to reproduce defects and provide as much detail as possible to improve the chances of reproducing it, the team as a whole needs to buy in that at times, thereproduction may not be worth the effort and it might be a better use of everyone’s time to jump into fixing the issue directly through code reviews and tracing code paths.

About the Author

QA InfoTech

QA InfoTech

Established in 2003, with less than five testing experts, QA InfoTech has grown leaps and bounds with three QA Centers of Excellence globally; two of which are located in the hub of IT activity in India, Noida, and the other, our affiliate QA InfoTech Inc Michigan USA. In 2010 and 2011, QA InfoTech has been ranked in the top 100 places to work for in India.

Related Posts