Prototype mobile applications built with IBM Worklight for IBM Watson

How an IBM team extended a healthcare user interface for IBM Watson from desktop to mobile


IBM Watson is assisting oncologists in cancer research and treatment. For example, Watson can suggest confidence-weighted treatment options and accelerate treatment identification, publication, and distribution. The IBM Watson solutions team developed a sophisticated browser-based application that not only presents IBM Watson query results to oncologists but also facilitates their patient-consultation workflow. The IBM High Performance On Demand Global Solutions (HiPODS) Enterprise Mobile team has now extended this solution with full mobile support. Using IBM® Worklight® as the supporting infrastructure, the team created a prototype for a sophisticated mobile app to complement the browser-based application.

The hybrid app-development model in IBM Worklight gave us many advantages in prototyping. But it also presented challenges in our efforts to make the app perform as well as a native application. This article describes how we solved these problems to produce a highly responsive mobile UI. Supported on the Apple iPad and Android tablets, the mobile application is suited to the fast-paced workflow of a hospital environment.

In addition to describing the architecture, design, and implementation of the mobile prototype, this article discusses key issues for mobile application developers, especially those working in the medical field and beyond. This article focuses on how mobile technology can enhance the effectiveness of IBM Watson by providing access beyond the desktop. Find details about IBM Watson technology at the IBM Watson information center (see Related topics).

Full solution architecture

The mobile prototype is part of an overall solution that meshes with oncologists' daily workflow in a hospital environment. In that workflow, a doctor reviews a patient electronic medical record (EMR) in a browser — typically, at a nurses' station — before seeing the patient. After seeing the patient, the doctor dictates the diagnosis, the prescribed treatment, and any lab tests needed. Eventually, the dictation is transcribed to text, and the doctor reviews the transcription for accuracy. To fit into this workflow, the browser-based application enables a clinician to view case details at the desk before seeing the patient.

The mobile application prototype runs on a tablet so that the doctor can refer to the patient EMR and query IBM Watson while seeing the patient. For consistency, the views of the mobile application are similar in content, look, and feel to those in the browser-based application. But the doctor can also interact with the mobile application with gestures such as swiping or pinching. In addition, the mobile app supports the tablet's audio and camera features. The doctor can add to the EMR by dictating directly into the tablet or using the tablet's camera to take photos of the patient.

For the hospital that we worked with, the full solution was envisioned as an integrated environment that includes customer components, the IBM Watson pipeline, and a presentation layer. Figure 1 shows the high-level architecture of a potential solution:

Figure 1. High-level solution architecture
Diagram of a high-level solution architecture
Diagram of a high-level solution architecture

Let's example the architecture shown in Figure 1 by going counter-clockwise, starting in the lower left. In the lower left, the Active Directory, EMR (and its associated database), and Treatment (and its associated database) components represent the existing services at the customer site. In the lower right, the Watson corpus and Watson pipeline components constitute the back-end services for IBM Watson. (The IBM Watson pipeline — in reality a complex subsystem — appears as a simple "black box," because it is not this article's focus.) The Cases and Attributes database holds additional data that's collected by the doctors to complement the IBM Watson results. An extensive REST API that the IBM Watson Solutions team developed serves as the interface between each application (browser-based and mobile) and all of the back-end services. The Watson pipeline and corpus communicate with the REST services through a Java™ interface.

In the top right of Figure 1, the browser-based application connects to IBM® WebSphere® Application Server for authentication and to the IBM Watson REST services to query data. The mobile application, depicted in the top left, connects to the IBM Worklight server. The Worklight server hosts both the runtime for the application and adapters for connecting to the same directory server and IBM Watson REST services that the browser-based application uses. Audio and photo files captured by the mobile app are stored in the back end through the Java Content Repository component.

Authentication and authorization

Because the browser-based application is hosted on WebSphere Application Server, it uses standard WebSphere user authentication and authorization. When the user logs in, the application server authenticates with the hospital's enterprise directory service (such as the IBM Directory Server or Microsoft Active Directory). For authorization, the server manages the user roles through its own Users and Groups feature (which the customer preferred over adding new attributes to its enterprise directory). The web application checks the assigned user roles and enforces restrictions on particular roles. The REST API is hosted on the same application server. The REST URLs are secured by URL pattern and role in the server's web.xml file. When the server receives a URL request, the server compares the user in the request to the server's user-to-role mapping to verify that the user is authorized to access that URL.

The mobile application running on a tablet uses the same mechanism by authenticating with the directory service to obtain a session token. If the user has no record in the directory, the application receives an unauthorized to access error message. If authentication is successful, the token that is received is used in the REST URL to access the back-end services.

Worklight adapters

The IBM Worklight adapter is the transport layer hosted in the IBM Worklight server for connecting to the back-end systems. The adapter can be used to retrieve information and perform actions on database, REST service, and other components. Three kinds of adapter — SQL adapter, HTTP adapter, and IBM® Cast Iron® adapter — are available. An adapter is implemented with an XML file, a JavaScript file, and an optional Extensible Stylesheet Language (XSL) file. The XML file defines the connectivity information and the procedures that are exposed to the mobile application. The JavaScript file defines the implementation of the procedures. The optional XSL file stores the transformation scheme for processing raw XML data. The data that is returned from the adapter is in JavaScript Object Notation (JSON) format. The JavaScript callback function processes the response from the procedure.

For the prototype, we use the HTTP adapter to connect to the back-end REST services. We define an adapter for each REST service implementation. We also define an adapter-helper JavaScript file that calls the adapters' procedures and defines the callback functions to handle returned data and error conditions. This helper file encapsulates the association between adapter and procedure. Therefore, the JavaScript files that support the mobile application view can call the function in the helper file without knowing which adapter is being used.

The mobile application's design

A unique characteristic of mobile applications built with IBM Worklight is that they are implemented with HTM5, JavaScript, and Cascading Style Sheets (CSS) — all standard technologies for web development. Worklight app design can therefore use the same best practices as the design of a web application. The prototype's design reflects a current web-design trend: enhancing the user experience through a single-page application (SPA) model, as opposed to the older Web 1.0 and Web 2.0 models. Because IBM Worklight supports HTML5 in hybrid mobile applications, we were able to adopt SPA in our design.

In the Web 1.0 model, applications are intended for navigating among web pages. The application typically advances to another view when the user interacts with page elements (such as hyperlinks and form-submit buttons), causing the browser to issue a request to the server for a completely new page. As the new page's data is retrieved from the server, the old page is unloaded and the new page is rendered to the screen. Network latencies can cause the user to experience perceptible disruptions while the page loads. And unchanged page elements such as navigation, toolbars, and images are retransmitted unnecessarily over the network when a page reloads. This design is easy to implement, but the more complex the UI, the more slow and awkward the user's interaction with the application tends to be.

In Web 2.0 applications implemented with Ajax (asynchronous JavaScript + XML), individual elements in the web page can change without requiring the whole page to update. The smaller granularity and asynchronous activity improve the user experience, but the design requires more implementation work on both the server and client sides.

Today's highly capable browsers and powerful hardware make the modern SPA model possible. Most of the processing of the web application — HTML rendering, data processing, and business logic — is pushed to the browser on the client side. In a SPA session, the application views and logic are retrieved in a single page load. The client communicates with the server only for authentication or to synchronize data. If any further server requests are sent, the processing for these requests occurs in the background. All user interaction and changes to application state are handled within the context of a single web document. Network latencies are less perceptible, and the users' experience is as fluid and comfortable as their experience with native applications — whether they're using a desktop browser or a phone app (even on a sketchy 3G connection).


With the SPA design, the application views are retrieved in a single page load, and the data is requested from the server at a later time. Therefore, we use a Model-View-Controller (MVC) architecture to separate the data layer from the presentation layer. MVC is an architecture for efficiently relating the UI to underlying data models. As the name suggests, an MVC design consists of three components:

  • The model consists of application data and business rules.
  • The controller mediates the input, converting it to commands for the model or view.
  • The view can be any output representation of data, such as a chart or a diagram. Multiple views of the same data are possible, such as a pie chart for managers and a tabular view for accountants.

The central concepts behind MVC are code reusability and separation of concerns. Figure 2 illustrates the relationship among the components in an MVC design:

Figure 2. MVC design pattern
Diagram of the interaction among the user, model, view, and controller in an MVC design pattern
Diagram of the interaction among the user, model, view, and controller in an MVC design pattern

Because the views are loosely coupled with the data model, MVC is well-suited for the SPA architecture. You can create views without needing to deal with the data model's underlying logic. Figure 3 shows how we used the MVC design in our mobile application and integrated with the adapters in the IBM Worklight server:

Figure 3. MVC in a mobile app
Diagram showing implementation of the MVC design in a mobile app using HTML5 and JavaScript
Diagram showing implementation of the MVC design in a mobile app using HTML5 and JavaScript

Development environment

Developed with IBM Worklight Studio as a hybrid application, the mobile app uses HTML5 and JavaScript together with the Dojo Toolkit, a JavaScript framework. With IBM Worklight, an app can target multiple mobile platforms (iOS, Android, Microsoft Windows Mobile, and Blackberry) with a single development effort. HTML5 also enables us to deliver a rich visual interface that's comparable to that of a native mobile application.

Because HTML5 does not have access to native mobile device features such as GPS, camera, audio, and contact list, IBM Worklight uses Apache Cordova to add native support. See Native device support later in this article for details.

Dojo Toolkit

The SPA and MVC architecture enabled a fluid and responsive hybrid mobile application. To add touch and native-theme support for mobile devices — and other features — we used the Dojo framework:

  • Dojo MVC provides the back end for view-to-model data binding (for example, view controller) on the client, which enables development of a data-rich UI.
  • Dojo Touch provides a set of events that are designed to work similarly in a desktop browser (for example, with the mouse) and touch-screen devices. The developer can focus on coding for one set of events instead of dealing with mouse and touch events separately.
  • Dojo Theme provides theme support for multiple mobile platforms.
  • Dojo Mobile widgets provide common scroll panels and a list menu, matching the native device look and feel that users are accustomed to.

Major components

The following key components implement the SPA architecture:

  • router.js:
    • Preloads views
    • Listens to views loading and uploads messages
    • Executes view-specific initialization and deposing modules
  • domModifier.js
    • Handles and updates common Document Object Model (DOM) elements, such as toolbar and menu
    • Handles loading screen and executes callbacks after loading
  • connector.js
    • Calls REST services
    • Validates data
    • Packages data models for views to consume

The design is general in nature and can be reused in other mobile applications.

Custom-widget fly-in panels

When a doctor views patient data with either the browser-based application or the mobile app, a large amount of information is displayed from the patient EMR and the IBM Watson results, including the advice and evidence. The browser-based application can rely on a large monitor to display the data, and page navigation can use a tree directory structure. On a tablet's smaller screen, navigating the same amount of information is more challenging. In many cases, the user wants to compare results from two or more panels that might be in different branches in the tree.

To solve this problem, we designed a custom widget that can open multiple views concurrently, which enables the user to navigate the views easily. The widget renders and loads any HTML page in a single API call. It is self-contained and aware of device screen size. Designed for a mobile device, it handles all gesture control such as swiping to scroll and pinching to close. It tracks all panels that are currently open and handles the layout and animation automatically.

The widget is reusable in other applications that require a large amount of information to be displayed.

DOM manipulation structure

In the SPA design, two factors affect the manipulation of DOM elements in the web page. First, because views are constantly being swapped in and out of the main screen, DOM elements are constantly being updated when new views come into the main screen. Second, during the development of a complex enterprise application, several team members might be developing different views concurrently. For these reasons, a structured approach is needed for multiple developers to manipulate the DOM elements so that no two views try to update a DOM element at the same time.

In the prototype, we used an event-based structure for DOM manipulation. The controller serves as the controlling component to keep track of all the views and as the single point of entry for DOM manipulation.

Each view registers a predefined start-up and clean-up function with the controller. The controller executes the corresponding start-up or clean-up functions when a swap-in or swap-out event is raised from the views. All the DOM-manipulation rules are defined in the start-up and clean-up function. This DOM-manipulation structure enables large-team collaboration, which makes team development more efficient and debugging easier.

Native device support

Mobile devices offer several capabilities that are unavailable on typical desktop computers. To take advantage of some of these capabilities and complement the browser-based application, the mobile app adds audio and camera support. Doctors can dictate messages to IBM Watson through the tablet app and use the tablet's camera to take photos of the patients they are treating. Initially, we introduced these features in the Rating view for each treatment option, but it can also be used in other views as applicable.

In the Rating view, the doctor can rate the relevance of a treatment. When the doctor taps the stars icon of the possible treatment options, the Rating pop-up window appears for the doctor to select a rating and enter a comment. The doctor can type a comment, tap the microphone icon to record a message, or tap the camera icon to take a picture.

If the doctor has previously typed a textual comment for the particular treatment, the existing comment is displayed and can be edited. In this case, the replay and display icons for any existing recorded dictation and photos appear alongside the textual comment. The doctor can delete those audio or photo files and recapture them. For a given treatment, dictation and photos from all users are displayed, but users can update only their own dictation and photos.

Currently, the dictation and photos can only be played back (dictation) or displayed (photos) for reviewing purposes. In the future, we can consider using an automated speech-to-text system or a human transcription service to convert the audio into text, which can then be used as input to IBM Watson. In that scenario, the transcribed text will be displayed as text in the comment section.

Audio and camera

The mobile application uses Apache Cordova 2.2 to access the native device features and to ensure that the support is portable across mobile platforms. This is part of the standard IBM Worklight Studio support and enables portability of the app between iOS and Android devices.

For audio, we used the Cordova API media object, which enables recording and playback of audio files on a range of mobile devices. Similarly, for camera support across mobile platforms, we used the Cordova camera.getPicture API. This API is used either to take a photo with the tablet's camera or to retrieve an existing photo from the tablet's photo album. The image is returned either as a base64-encoded string or as the URI of an image file.

Content repository

The audio and photo files are captured in the device's local storage. To store and manage these files permanently in the back end, we used a content repository based on the Java Content Repository (JCR) specification. This approach enables the app to work with any JCR-based software such as Apache Sling or IBM Web Content Management. We used Apache Sling, which includes the Apache Jackrabbit product.

The JCR specification provides the following functionality:

  • Abstraction of diverse data storage: small and large, structured and unstructured, binary, metadata, and relationships
  • Generalized content services: access control, locking, versioning, transactions, and observation
  • Separation of real storage from application interactions
  • A standard API

Figure 4 shows the JCR model, which includes:

  • Workspaces
  • Hierarchy of nodes
  • Properties associated with a node
Figure 4. Java Content Repository model
Diagram of the Java Content Repository model
Diagram of the Java Content Repository model

Apache Sling, which is based on the JCR specification, includes:

  • OSGi - Sling is built with OSGi bundles and uses OSGi core and compendium services.
  • Web API - This API for content-based web applications extends the Java Servlet API and provides more functionality for work with content.
  • Request processing - A request URL is resolved to a resource. Then, on the basis of the resource only, Sling selects the servlet or script to handle the request.
  • Resources - These represent the resources addressed by request URLs.
  • Servlets and scripts - Servlets and scripts are handled uniformly as resources themselves and are accessible by a resource path.
  • Launchpad - Sling uses a thin launcher to integrate with existing servlet container, launching Sling as a web application or providing a main class to represent a standalone Java application.

Integration with the mobile application

Figure 5 shows the full infrastructure that supports audio recording and photos:

Figure 5. Integrating audio and camera support to mobile app
Diagram showing the integrating of audio and camera support to mobile app
Diagram showing the integrating of audio and camera support to mobile app

The audio and image files are initially stored in the device memory. Then they are transferred to the JCR in the back end for permanent storage. The mobile application initiates an HTTP POST directly to the JCR REST service with the audio or photo data as content, thereby improving performance by avoiding an extra hop through the adapter. The data is stored as objects in the JCR that can be accessed with URLs. Because the JCR API requires authentication, the app uses the same authentication mechanism as the browser-based application. When the JCR returns the URLs pointing to the new objects in the repository, the mobile application calls the IBM Worklight adapter to save the URL in an existing IBM DB2® table. The objects are associated with the user so that the user can retrieve them later.

When the Rating view is displayed, the mobile app uses the IBM Worklight adapter to obtain the textual comment and the audio and photo URLs from the table. The text is displayed in the comment box, and the audio and photo URLs are displayed as icons for playback or preview. When the user clicks a playback or preview icon, the application sends an HTTP GET to the JCR REST service to obtain the audio or photo file.

Cordova supports uploading files from a mobile device to a file server and downloading files from a file server to a mobile device. For example, the Cordova code segment in Listing 1 uploads audio and photo files to the JCR server:

Listing 1. Sample Cordova code segment for file upload
// Set options
var options = new FileUploadOptions(); 
options.fileKey = 'audio';  // or 'photo' for photos
options.fileName = filename;
options.mimeType = "audio/vnd.wave";  // or "image/jpeg" for photos
options.chunkedMode = true;
// Set security header
var headers = {Authorization:'Basic YWRtaW46YWRtaW4=');
options.headers = headers;
// Upload photo or audio file
var transfer = new FileTransfer(); 
transfer.upload (localFileURI, serverURI, success, failure, options);

Issues and considerations

In developing a mobile application for healthcare, we encountered various issues that remain open questions for future consideration.

Personal health information

Because healthcare use cases can involve the handling of personal health information (PHI), security is a critical issue. A mobile device brings flexibility but also adds potential complexity to security management. Some areas of focus include:

  • Controlling access to the app: The typical user login can be used, but depending on customer policy and government regulation, additional security measures might be required. For instance, the back end might validate the device ID or subscriber ID.
  • Transmission of PHI over the air: Encryption (which should meet the client's standards) might be needed.
  • Storing PHI in mobile device local storage: Locally cached data enables users to use the application while offline, and updates can be made to the back end when the device is online. The customer needs to choose a security policy concerning local storage. IBM Worklight features the JSON store, which is encrypted on the device storage.
  • Managing the devices: The enterprise might need to enforce security policy such as password rules, or disable access if the device is lost. IBM® Tivoli® Endpoint Manager for Mobile provides this support.

The current prototype uses simple login to control access. A complete solution that addresses PHI security must take into account the customer's policies and its procedures for addressing government regulations that relate to PHI.

Browser and mobile application consistency

Enterprises typically have browser-based applications that have been in use for a long time. The UIs of mobile apps that are introduced later must be consistent with those of their browser-based counterparts so that users can switch between the two without confusion. The basic theme — such as color scheme, graphics, and layout — can be managed through the appropriate CSS in the IBM Worklight project. But in many cases, the browser-based application and the mobile app do, of necessity, offer different functionality. For instance, data entry can be easier in a browser-based application and more limited in a mobile app. Viewing results can be more natural on a mobile application with gesture control for scrolling and zooming.

Mobile applications tend to focus on a particular use case rather than replicate the functionality of the browser-based application. For the applications in this project, we ensured that the theme and the page navigation are similar so that users can easily find what they are looking for. To optimize the user experience, the applications do offer different ways for the user to interact. In the mobile app, users can swipe, pinch, push up, and push down. This approach seems to facilitate the best use of the mobile device, but this is an area in which more experimentation is needed to identify the best practices.

Working with open source code

Recent trends show that adoption of open source code in production applications is increasing. In IBM Worklight, Apache Cordova is a good example of successful use of open source software. In the prototype app, we also used Apache Sling for managing audio and photos. In the best case, open source code increases productivity through code sharing. Mature open source software that is supported by a large community can become a stable platform for production use. But in other cases, support and security can be an issue.

Some enterprises, such as banks, might enforce IT policies that severely restrict the use of open source software. In addition, an open source developer typically relies on other open source components, which can create unexpected dependencies. In one case, a customer was using a Cordova plugin — available as open source code — for scanning QR code. The mobile project failed to build because libraries were missing from a chain of dependencies, and it took some effort to find all of the correct open source components. Better tools are needed for managing open source dependencies and versioning. Some best practices from Eclipse and Linux can serve as a model.

Dojo Mobile and jQuery

One of the first decision points in mobile application design is the UI framework to use. IBM Worklight Studio supports development with both Dojo Mobile and jQuery. The browser-based and mobile applications for IBM Watson were both built with Dojo. We believe that this is the best practice, but our team also experimented with jQuery. The following observations compare the two frameworks:

  • MVC: Dojo Mobile offers more built-in support for implementing the MVC design pattern, whereas jQuery requires an additional framework such as Backbone. But from a different perspective, jQuery can be considered more flexible, because the developer can choose the best components.
  • UI appearance and user interaction: Dojo Mobile and jQuery offer similar performance in this area.
  • Software maintenance: Many jQuery features depend on third-party plugins. As new versions of these third-party components are introduced, dependencies among them might be broken. Dojo Toolkit offers a more comprehensive library for enterprise web applications, with fewer third-party dependencies.
  • Development support: jQuery has a larger community, and support is easy to find. jQuery is excellent for building web pages and standard web applications. But Dojo widgets are built to accommodate enterprise web applications, and Dojo offers a solid, flexible building block for our applications; its functions are easily extensible.


In this article, we presented the full design and implementation of an advanced mobile prototype for IBM Watson. We put the IBM Worklight Studio development environment to full use and leveraged the Dojo Mobile framework, the Cordova library, and other open source components. The mobile application complements the browser-based application by enabling a doctor to access the IBM Watson solution while seeing a patient or when away from the office. We solved a number of problems in implementing the hybrid mobile application to make it as responsive as a native app. The mobile application also adds mobile device support to give the doctors new tools to improve productivity in their normal routine. We hope that the reusable assets, findings, and lessons learned in this project can be useful for other teams developing mobile applications.


We acknowledge the major supporters of this project:

  • Angel Diaz, Willy Chiu, and Larry Hsiung of the Cloud Labs and HiPODS management team
  • Our colleagues Rahul Jain, Chris Kiernan, Chin Huang, Raymond Wong, and Ted Hao of the Cloud Labs and HiPODS team
  • Rob High, Bill Hume, Jeff Eisner, and Jayashree Subrahmonia of the IBM Watson management team
  • Matthew Sanchez, Ed Seabolt, Andrew Ortman, Jon Richter, and Michael Holmes of the IBM Watson solutions team and client team

Downloadable resources

Related topics


Sign in or register to add and subscribe to comments.

Zone=Mobile development, Java development
ArticleTitle=Prototype mobile applications built with IBM Worklight for IBM Watson