Now you see it….Now you don’t! How many times have you followed through a set of steps to accomplish a task using a software or website and ran across intermittent defects? These mystery defects can be engineers’ nightmare, let alone the end users.
Before we begin to document the defect, we have to perform a repeated set of steps to drill down to the core cause. The objective of repeating the steps is to find a pattern in the undesired behavior in the system. Sometimes repeating the steps may not behave consistently. Let’s look at some of the factors that can play a role in troubleshooting intermittent defects:
a) Repeating the steps, but with different browsers, and/or operating system.
b) If it’s a form, what you enter matters a lot. So keep track of what data you are feeding in the system, such as alphanumeric characters, special characters, double-byte characters, or combination of these characters.
c) Looking under the hood. By that I mean tracing web server logs or validating what gets stored as you step through the functions on the user interface.
d) Process of elimination. When there are multiple servers involved in the web topology, the steps to reproduce will need to be repeated on each server separately with one server running at time.
e) Disabling/enabling cache to ensure that data is refreshed at a consistent rate or as needed.
f) Browser cookies. I usually start my browser session with a clean slate to eliminate stale browser cookies. For someone like me, who traverses through development, test and production environment several times a day, it’s essential to avoid any browser session conflict.
g) Take a holistic approach. When the system is composed of multiple applications, it might be advisable to test the problematic area in conjunction with testing other applications.
h) Checking the server health after prolonged usage of the system to determine if there’s a memory leak. Performing stress test to cause the system to reach an unstable state can help determine the system’s threshold for handling the load.
These are just some of the techniques we use here at developerWorks to find the root cause of “ghost” defects. These defects are the ones that need lot of patience and teamwork to find the root cause & implement a sound solution.
If you have any particular method that have helped you in dealing with intermittent bugs, please share them.