In order to tackle this problem, an Automation Engineer cannot simply look at it from a “one size fits all” perspective to create a set of objects and re-use them across all combinations of platforms. For example, there are fundamental differences in how an app behaves on iOS and Android, even with something as basic as a “back button” has its quirks.
Although these fundamental differences can be grouped together as a step or action, they are unique enough to not be able to simply share an object between the two OS’s.In some cases with mobile testing, you may be able to get to the object-level, however this usually requires that you instrument your app, or test on an emulator. While this fulfills a piece of your testing matrix, you will probably need to seek a couple tools to get this done across all platforms. In other cases, the content you are testing might be HTML-based and you can test by WebKit profiling. Again, part of your testing matrix is fulfilled, however you aren’t quite there.This may be enough to satisfy a short-term goal, but at some point you need to be testing on real mobile devices.
In order to truly automate on mobile, your mobile testing “Utility Belt” needs to be designed in such a way that allows for testing by object when possible, element when possible, and also be able to quickly fall back on text or image verification in order to satisfy all areas of your testing matrix and assure the highest quality of your mobile product. Having the flexibility to be able to choose how to get the testing done is paramount since as an Automation Engineer,you very rarely have a say in how a particular app or mobile web site is developed. The job requires you to sometimes understand functionality without necessarily being privy to the construction, and there is always a tight timeline to achieve results. The right tool for the job is a tool that takes all of this into consideration, and provides a platform to consolidate all of these different types of testing approaches.The first step is to determine the type of app you are testing. Is it fully native, fully web, or somewhere in between?
- If it’s fully native, you may be able to get to some objects on a per platform basis, but you will probably be falling back on text and image based verification, especially if you are trying to go cross-platform.
- If it’s fully web, a lot of testing can be done up front in a WebKit profiler. When it comes to real devices, element-based testing can be done cross-platform if you want to instrument, or you can fall back on text and image verification.
- If it’s somewhere in between, you’ll need to mix and match.
Note: This article was recently published in the April, 2013 issue of Automate Software Quality Magazine.