Add IBM Mobile Quality Assurance to your mobile quality regimen

If you want to make your interactive manual testing of mobile apps significantly more efficient and effective, try IBM Mobile Quality Assurance. MQA is a hosted Software-as-a-Service capability that helps you continuously deliver high-quality applications. And it's extremely easy to start using MQA in the pre-production and post-release phases of your mobile app lifecycle.

Share:

Leigh Williamson, Distinguished Engineer, Software Group , IBM

author photoLeigh Williamson is an IBM Distinguished Engineer who has been working in the Austin, Texas lab since 1988, contributing to IBM’s major software projects, including OS/2, DB2®, AIX®, and WebSphere Application Server. He is currently a member of the IBM Rational software Chief Technology Office team, influencing the strategic direction for products in the Rational brand and leading the solution definition for mobile application development. Leigh holds a B.S. degree in computer science from Nova University and a master's degree in computer engineering from the University of Texas.



04 March 2014

Also available in Chinese Russian

Getting started with IBM Mobile Quality Assurance services | 6:29

Video: Getting started with IBM Mobile Quality Assurance
                    services
Transcript

Overview

Several techniques for testing and validating mobile apps are in use today. The IBM point of view is that an effective mobile development project will employ all of the applicable techniques, because each technique has its strengths and weaknesses. The various available techniques are not mutually exclusive. The most effective mobile quality strategy balances the use of all forms of mobile test execution and rolls up the results from each into a comprehensive overall mobile app "quality metric."

IBM Mobile Quality Assurance is designed to fit well as a component of a comprehensive mobile app quality strategy. Such a strategy should include running a battery of automated mobile tests against each build of the mobile app to ensure that it meets minimum quality levels before employing IBM Mobile Quality Assurance services to distribute the build to human interactive testers, so that bugs and crashes can be effortlessly captured in rich context.

WATCH:Webcast: Continuous Quality for Mobile Apps—Reduce Your Time-to-Feedback

Over-the-air distribution

When a new build of the app passes the initial battery of automated tests, the developer can upload the app binary (.apk or .ipa file) to the IBM Mobile Quality Assurance service and identify the people who should be notified about the availability of this new build. This set of mobile app evaluators/testers receives an email notifying them about the new build. And when the tester clicks a link in the notification email, the new version of the app is automatically downloaded to their mobile device and installed, ready for immediate testing. The mobile app testers can be confident that they have the correct build of the app to be tested.

In-app bug reporting and feedback

As testers perform interactive manual testing of the app and encounter a defect of any kind, they can use IBM Mobile Quality Assurance in-app bug reporting capability to submit a defect right from inside the app being tested on their mobile device.

The tester simply shakes the mobile device and the app being tested goes into "bug reporting mode." This mode suspends normal behavior of the app and allows the user to capture one or more screen images from the mobile app. The tester can annotate the screen image with lines, circles, arrows—anything that fingers can draw.

After the screen image is captured, the tester is presented with a text box to be used to describe the defect. Once the description of the problem is entered, the tester taps on the Report button, and the defect information is sent over the network to the IBM Mobile Quality Assurance service. Along with the explicit information (screen images and text description) from the tester, rich technical details of the context of the mobile app and the device on which it was running are captured and sent as well.

Some of the rich context for each defect captured includes:

  • Mobile device type
  • Mobile operating system and release level
  • Network in use, including carrier or wireless details
  • Memory available and in use on the device
  • Logging output up to the point when the defect is reported
  • Battery level

This detailed technical information is invaluable for helping the mobile app developers troubleshoot the defect and understand the root cause of the problem.

Crash analytics

In addition to the in-app bug reporting capability of IBM Mobile Quality Assurance services, every application crash is captured by the service logic. Each time the application crashes, the entire context of the application and the device on which it was running are captured at the moment of the crash. This critical, "must gather" data is sent over the network to the IBM Mobile Quality Assurance services, where it is analyzed and made available to the development team.

If you use IBM DevOps services for defect and work item tracking and management, you can configure IBM Mobile Quality Assurance services to automatically open work items for each crash or bug report that comes into the service.

You can leverage the crash data capture capabilities of IBM Mobile Quality Assurance services during the initial automated battery of tests on the app, during the manual interactive phase of testing, and even after the app has been released to the app store.

Additional analytics within the service recognize crashes that occur at the same spot in the mobile app and aggregate them so that you can see how many times a crash occurs at the same location in the app logic.

Quality is in the eye of the end user

The main focus for quality assurance of mobile apps is on the pre-production phases of the development lifecycle. However, quality assessment should not end when the app gets released into production. There is still very important data about the behavior of the mobile app "in the wild" that can be obtained and used to help the developers continuously improve the app.

It isn’t practical to expect that every conceivable defect in a mobile app can be caught and fixed before it is released into production. Even after they are deployed to the app store and installed on end users' devices, the best mobile apps continue to capture context information for every crash that occurs and deliver that information back to the app development team.

Besides outright crashes, it is valuable to solicit feedback from your end users about how they perceive the mobile app. Most popular mobile apps include some kind of "in-app feedback" mechanism so that users who would not take the time and effort to write a review in the app store can at least send the development team a short message about how they view the app. And it is especially important to capture the context of such feedback at the time it is submitted, so that the development team can know if special conditions are contributing to the impression conveyed in that user's feedback.

Sentiment analysis

App store review comments offer a rich source of quality assessment also. If your mobile app has only a few dozen reviews in the app store, it is easy enough to read each one and gain insight into the sentiment of your users. But once the number of reviews reaches several dozen or more, you need an analysis tool to effectively and efficiently mine the key insights from that amount of data.

IBM Mobile Quality Assurance includes an app store review analysis capability that captures all of the review text and searches for a set of special "user sentiment" keywords in each review text. Analysis for app store reviews is organized into 10 distinct "attributes" about your mobile app, such as usability, stability, performance, and elegance.

You can drill down into each of the user sentiment attributes for your mobile app and see the analysis used to produce the score for that attribute. You can even see the specific list of reviews containing comments about that attribute.

This app store user sentiment is invaluable, especially when you correlate the sentiment expressed by the reviewers with hard technical evidence in the crash reports and in-app user feedback records.

Continuous delivery and quality assessment for mobile apps

A best practice mobile app testing strategy strikes a balance between automated tests and interactive human-based testing.

 

The ideal quality cycle begins by running a battery of automated tests against the output of the continuous integration build process for your mobile app. Once this initial battery of automated tests has verified that the latest build meets minimum quality criteria, the build can be distributed to a group of testers and internal evaluators who will perform the interactive testing.

Diagram showing hourly, daily, and weeking testing in the mobile app development cycle

After the mobile app has passed the automated and interactive tests, it is a candidate to be released to production use and distributed to real end users (presumably via a public app store or private enterprise app store). Even after the mobile app has been released into production, you can continue to obtain quality assessment data about the application.

Conclusion

IBM Mobile Quality Assurance services are designed to play a part in all three phases of the mobile quality regimen, making your interactive testing as efficient as possible and capturing critical must-gather technical data about crashes and bug reports filed by your testers. And, as the reviews for your app build up in the app store, use the sentiment capabilities of IBM Mobile Quality Assurance to gain insight about how your end users perceive your mobile app.

This comprehensive mobile quality strategy will give your team faster resolution of crashes and bugs in your app, and deeper insight into the conditions in which the app runs in the real world. You’ll be heading toward the coveted 5-star rating in no time!

Resources

Learn

Get products and technologies

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Mobile development on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Mobile development
ArticleID=964505
ArticleTitle=Add IBM Mobile Quality Assurance to your mobile quality regimen
publish-date=03042014