Take control of your app feature rollout and measure the effectiveness

By: Srinivasan Nanduri

Take control of your app feature rollout and measure the effectiveness

Have you wondered why rolling out features in your app is so difficult and time-consuming with very little or no feedback on how the newly released feature is performing? I bet you have. Well, you have an answer now – introducing IBM Cloud App Launch Service.

Let’s say you have a popular app in a Mobile App Store which lets shoppers buy your merchandise using your popular app. You have users classified as Platinum / Gold / Silver /Regular based on their loyalty / membership and use either iOS / Android phones. Your buyers are complaining about complicated checkout process since they tend to forget user/password and unable to complete the transactions.

While you are exploring various other options to replace user/password authentication, you decide to conduct a small experiment –

  • Offer Touch ID based checkout feature (through a new button in the app) to all iOS users with Gold membership.

  • You decide to try two different colors for the button and measure if there’s any significant change in the feature usage.

  • Lastly, you know the upcoming biggest shopping day of the year is a great opportunity to test this hypothesis so you would want to perform the experiment only during that weekend.

To summarize you would want the following,

  1. Dark launch a new feature in your existing app without exposing it to everyone

  2. Define and implement a new feature and create its variants

  3. Create an audience that includes only Gold iOS users

  4. Select a window range when the experiment is conducted and in a real-time measure the effectiveness of this experiment during that window.

The newly launched IBM Cloud App Launch will help you perform all the above and much more. Let’s see step-by-step how you would accomplish all the above tasks in a simple and intuitive way.

Let us go one step at a time. Once you create an App Launch service instance you are landed on the Getting Started page. Quickly skim through the sections, but don’t sweat – this blog will walk you through all the steps.

Your mental model for App Launch service would look something like below,

  1. Setup an App Launch service instance

  2. Download and Integrate the App Launch `SDK` in your app

  3. Initialize your app

  4. Define a new `Feature`, get the feature keys

    • Develop the feature, update the feature keys in your app

    • Update the status of the feature to Ready in the App Launch console

  5. Build an Audience pool

  6. Create an `Engagement` campaign for `Feature Control`.

    • Define and customize one or more variants for this feature

    • Select an audience

    • Define and schedule the date/time when engagement should be active

  7. Push the new version of the app at least a month before the biggest shopping day of the year so enough users would have downloaded the modified (dark-launched) version from the AppStore.

First, visit bluemix.net and sign up for an account, if you don’t have one. Once logged in, visit the Catalogpage (link on the top right corner).  On the left pane, select Mobile as the category and click App Launch

Now, you are ready to create your first App Launch experiment!

SDK

The App Launch Client SDKs are currently available for Android and iOS.  Integrate the SDK into your app. Download and install platform specific SDKs from here.

The ‘readme’ document in the SDK has detailed information on installation.

In the above step, you accomplished Step 2 and 3 of your requirement

Features

Select Features on the left-side pane. Let’s get familiar with what a Feature is. A Feature in App Launchservice is a simple grouping of attributes and metrics. Create a new Feature by following the below steps –

  1. Create a feature called, Touch ID Button

  2. Add a feature property, called Button Label and set value to Touch ID Checkout

  3. Add another feature attribute, called Button Color and set default value to #AD343E

  4. Add one more feature attribute, called show_button and set value to true

  5. To track metric for this feature, add a metric, called TouchID Clicked

In the above step, you accomplished Step 4 of your requirement.

Audience

Once Feature creation is done, define Audience. On the left pane, select Audience.

The Audience in App Launch is how you build a segment of users on whom you would like to target this new feature. Follow the steps below –

  1. New Audience Attribute

    • Add a New Audience attribute, called customer type

    • Set customer to be a string

    • Under Description enter, gold

  2. New Audience – Show this feature to only gold and iOS users

    • Add a New Audience, called Gold iOS users

    • Under Attributes, Select the pre-defined Platform attribute as iOS

    • Now, Select the attribute customer type that you defined

    • Check gold in the displayed customer type list

    • Select Save

In the above step, you accomplished Step 5 of your requirement.

Engagements

Once a Feature is created and Audience is defined, it’s time to bring all of it together in an Engagement. An Engagement in App Launch brings together one or more Features and Audiences together and allows you to experiment by creating multiple variances. Follow the below steps,

  1. Create an Engagement, called Holiday Button

  2. Select Experiment Mode as A/B Testing

  3. Select the feature you would like to A/B Test. From drop down select Touch ID Button

  4. To A/B Test Button color for feature Touch ID Button select,

    • Variant 1, set button color as #AD343E

    • Variant 2, set button color as #846075

  5. Select an audience – Gold iOS Users

  6. Specify Reach % for each variant, for example, 50% each

    • Variant 1, set Reach to be 50%

    • Variant 2, set Reach to be 50%

  7. Specify a Start date and End date. If start date is empty, engagement starts immediately.

  8. Select Create

In the above step, you created an engagement along with accomplishing Step 6 of your requirement.

All that’s left now is to download the SDK, Feature Toggle, and Metric keys and start incorporating them within your TouchID logic. Let’s look at few code snippets. The Feature Details window has an option to export the feature as a JSON file which can be used in the client application to load the default values..

created an engagement

Also, ensure the state of the Feature is set appropriately. For example, if the Feature is set to Under Development then the Feature will be unavailable to be part of an Engagement. Ensure the Feature is set to Ready to be included in an Engagement.

Ready

App Launch SDKs offer a rich set of APIs to check whether a Feature is enabled or not, then get value for Feature keys. The below code snippet shows an Android sample to check for Feature and then get the label text.

App Launch

A fully working sample can be downloaded from App Launch git repo here.

Once coding is complete, go ahead and post your app to App Store or Play Store. As and when users download and use your new version of the app, your users will be receiving the dark-launched feature hidden in the code. The App Launch service SDK will be registering the users as they launch the app, but since the engagement is disabled until the week of the biggest shopping day of the year, no users will see the new TouchID Button. As intended, on the biggest shopping day the Gold iOS users will see a new TouchID-based check out button when they checkout their items in the app. Fifty percent of those users will see one button color and the other fifty the second variant button color. As each user buys merchandise by clicking the TouchID button, App Launch service collects such events and displays the effectiveness in real-time. Once the weekend’s sale is over the feature goes to a dormant mode as the engagement goes into disable mode.

Summary

In summary, the newly launched App Launch service offered you the ability to control feature rollout to a small set of audiences in multiple variants and triggered the feature during a small period. Now you get it – the above is a simple App Launch use case, but how the service can be used is up to your imagination. Check out additional material here. Post your queries to Stack over flow with tag ibm-cloud-applaunch.

Happy Coding!

Be the first to hear about news, product updates, and innovation from IBM Cloud