Editor’s note: Randy Sandberg is Pongos Interactive’s Chief User Advocate, and heads up our testing department.
“Fundamentally you can’t do your own QA, it’s a question of seeing you own blind spots.” ~ Ron Avitzur
If you look carefully at the quote above you’ll find a typo. The point being that programmers are not best suited to find their own bugs. Just like writers rely on editors to catch their “bugs”, programmers use testers for the same purpose. That is, it takes a combination of highly skilled programmers and testers to create a quality software product.
Within the development community, it’s a well known fact—given today’s complex software—one cannot test every aspect of a project. Typical budgetary and timeline constraints only enhance this fact. Thus testers, with the help of other stakeholders, decide where to focus their efforts. This is called managing risks. That is, a decision not to test something saves time and money but increases the risk of bugs getting released into the wild.
How does one decide what to test and more importantly what not to test? The exact formula is beyond the scope of this article, but with your users in mind, three questions should be answered. First, what devices/operating systems/browsers do the majority of your visitors use? Second, what current features are popular and what upcoming features have your users most requested? And finally, if a particular feature were to break, what would the impact be on your users and your organization?
These questions help form a “Do Not Break” list of devices, operating systems, browsers, and features to focus limited testing efforts on. Once a plan of action has been made, and testers know what to do via test cases, then the process goes something like this:
1. Tester finds a bug and writes a problem report. The report includes a descriptive title, what the tester expected to happen, what actually happened, detailed steps to recreate the bug, and any other noteworthy information including screenshots and videos.
2. Programmer reproduces the bug, attempts to fix the bug, and then sends the problem report back to the tester for regression testing. Regression testing helps show the bug was fixed without introducing any new bugs.
3. Tester repeats the original steps and, if the bug can no longer be reproduced, closes the bug. Otherwise the tester sends the report back to the programmer for further investigation.
Testing with your users in mind not only saves time and money during the project, but ultimately pays off over the long term as well given less support issues and increased user appreciation as well as engagement. Oh, and as for the typo mentioned earlier, here’s how the quote should look: “Fundamentally you can’t do your own QA, it’s a question of seeing your own blind spots.”