Testing, testing, testing, 1, 2, 3 testing (for mobile apps)
monica914 060000WAY8 Visits (3460)
You stay in this business long enough and you get to a point where everything old is new again. Seriously, it’s like the fashion industry. I always chuckle remembering my daughter’s shock that I, um, had also worn bell bottoms.
And this is exactly what I think about when I hear all the chatter about the fragmentation of testing for mobile applications. There are a billion device combinations. Yup. Hasn’t there always been? About 10 years ago, to help choose how to test and to make a point, I did a spreadsheet with all the platform combinations for a single, web-based product we were building. Because we supported 3 database vendors and 2 versions of each, all the browsers (2 versions) and a whole bunch of server OS’s, I quickly came up with about a half-a-million possible combinations to test. I’d have to say, I think as bad as the mobile device configuration problem is, it’s not quite that bad. So experienced teams will handle this risk the way we always have. Figure out a few high priority configs, test those like crazy and sanity test on another subset of configurations that completely depends on capacity. Cross fingers and move on.
So while all the noise seems to be about device fragmentation as the biggest challenge in mobile testing, I think the elephant in the room is some of the other issues. Consider the expectations for the turnaround time on mobile applications. That’s faster than most traditional teams can contain and requires coming up with new ways of working. There are special challenges to setting up the test environment. How do you get last night’s build onto your device (be it virtual or real)? If it takes you 2 days to figure out how to do that, well that will really impact that turnaround time. Once you have it there, you’re testing away, find a problem, how do you get screencaps attached to your defect report? If this is something you scratch your head about check out this beta.
Then, there’s the decision on what and how to test. In order to be fast enough, everyone turns to automation. This is a really good thing. But there are new decisions about what layer to automate, how to automate for some of the “special” things about mobile (what if I lose my cell tower right in the middle of some critical transaction on my banking app?). Do you have technology that helps you create those kinds of simulations? Are you relying on manual testers executing tests on the mobile device or are you automating the flows through your app with a mobile-specific test automation tool?
I definitely don’t have the answers to a lot of the questions for mobile. No idea how to effectively build test automation to test a GPS feature or using the camera in a mobile application (those are really special to mobile!). I also think the fast pace of development and the application of lean principles means that quality in a mobile world is way more than just testing.
I’m hoping we can talk about these new issues rather than rehash the old…. What is your team doing to solve these problems? Or are you stuck? Do you have other stumbling blocks to your efficiency in getting out quality mobile applications? What are your new ways of working to meet the challenges of building mobile apps? Comment here or tweet to me (@monical914). Let's have a conversation!