Old skills, new tricks: Unit testing OpenWhisk actions in a serverless world

Share this post:

serverless openwhisk hero imageIn the last two weeks, I’ve been working on a new scenario for our Logistics Wizard sample app. Logistics Wizard is an end-to-end, smart supply chain management solution. It showcases how to execute hybrid cloud, micro-services, and predictive data analytics in the real world.

Weather impacts business

The scenario focuses on how the weather affects a supply chain. This is old news but very real, as pointed out by this infographic about the retail industry. Consuming weather data in IBM Bluemix is easy with the Weather Company data service in IBM Bluemix. The service provides access to current and past observations, historical data, and alerts. All you need is a location on the globe—and voilà! You can start making decisions based on current and future weather conditions.

In such a scenario you would:

  • Listen for weather events and alerts
  • Analyze the events
  • Make recommendations for new shipments based on the weather conditions
  • Notify the retail store managers of new recommendations
  • Have retail store managers review and approve or reject the recommendations

This process needs to scale with the number of locations. The more locations, the more weather events to track and process. Some pieces of the process may also be dormant until there is a good reason to trigger them. This makes a good candidate for a serverless architecture—leaving the need for scaling to the system.

Retrieving the recommendations and converting them into decisions looks like your typical CRUD API calls on a backend (something at which serverless architectures also excel). Serverless is not only about reacting to events—unless you consider API calls as just another type of events.

serverless openwhisk flow chart

You know the tools

In Logistics Wizard, we have a module called the ERP simulator. It simulates an Enterprise Resource Planning backend, exposing users, products, shipments, distribution centers, and brick-and-mortar locations. When I worked on this module, we picked Loopback for the implementation. When we needed to do unit tests, we selected Mocha, Chai, and Istanbul as components of the unit testing framework, a set of commonly used libraries in Node.js.

With OpenWhisk, you write your actions with your language of choice. I often write them with Node.js. OpenWhisk is not about learning a new language. It’s about reusing your skills and applying them to a new exciting type of architecture—and fortunately this proved to work for unit testing too.

One example to rule them all

Let’s consider one action involved in this scenario as a starting point: Retrieve Recommendations. It reads the recommendations created after analyzing the weather conditions. The user interface calls this action.

The action connects to the database, finds the recommendations, and returns them. The source code for this action is available here. Its OpenWhisk entry point looks like:

function main(args) {
  console.log('Retrieve recommendations for demo', args.demoGuid);

  return new Promise((resolve, reject) => {
      (err, result) => {
        if (err) {
          console.log('[KO]', err);
          reject({ ok: false });
        } else {
          console.log('[OK] Got', result.length, 'recommendations');
          resolve({ recommendations: result });

Notice it uses the Promise API as shown in the OpenWhisk documentation for asynchronous actions.

The actual implementation connecting to the database and returning the recommendations is done in another function:

 * Retrieves recommendations linked to a given demo
 * {string} cloudantUrl - URL to the Cloudant service
 * {string} cloudantDatabase - Database where recommendations are stored
 * {string} demoGuid
 * callback - err, recommendations
function retrieve(cloudantUrl, cloudantDatabase, demoGuid, callback) {
  console.log('Searching index...');
  const cloudant = Cloudant({
    url: cloudantUrl,
    plugin: 'retry',
    retryAttempts: 5,
    retryTimeout: 500

  const db = cloudant.db.use(cloudantDatabase);'recommendations', 'byGuid',
    { q: `guid:${demoGuid}`, include_docs: true }, (err, result) => {
      if (err) {
      } else {
        callback(null, => {
          // remap the recommendation object
          // to make it look like what we returned in recommend.js
          const recommendation = row.doc.recommendation;
          recommendation._id = row.doc._id;
          return recommendation;

This function receives the information of the Cloudant database, the identifier of the demo session we are dealing with, and a callback to publish its result. Then it searches Cloudant and returns the data after a bit of formatting.

Put the action to the test

What do I want to test in this action? The minimum would be to validate that:

  • The action is correctly returning recommendations stored in the database
  • It handles Cloudant failures properly

Here is how I tested the normal case:

  it('returns existing recommendations', (done) => {
      .reply(200, {
        rows: [
            doc: {
              _id: 0,
              _rev: 0,
              recommendation: {
                fromId: 10,
                toId: 20
            doc: {
              _id: 1,
              _rev: 0,
              recommendation: {
                fromId: 30,
                toId: 40

      demoGuid: 'MyGUID',
      'services.cloudant.url': 'http://cloudant',
      'services.cloudant.database': 'recommendations'
    }).then((result) => {
      assert.equal(2, result.recommendations.length);
      assert.equal(0, result.recommendations[0]._id);
      assert.equal(1, result.recommendations[1]._id);
      assert.equal(10, result.recommendations[0].fromId);
      assert.equal(40, result.recommendations[1].toId);

This test mocks the Cloudant query results—I don’t want to have to set up a Cloudant service for a unit test! Then it validates the expected values, matching the data in Cloudant, and are returned.

Keep them small and focused

This may sound obvious to most, but it is worth mentioning again: If you want to make sure you can easily unit test your actions, keep them small and combine them with OpenWhisk sequences. Multiple pages of code in one big JavaScript function will likely not be simple to test. Splitting this code in individual functions and actions will achieve better test coverage.

Detach yourself from the serverless context

When you run an action in OpenWhisk, there is some context information provided to you, such as the action arguments (e.g., the detected changes in a Cloudant trigger). You could pass these everywhere in your code but I found it is better to clearly identify which parameters each sub-function really needs and to give them only those. It also helps to test the individual functions making your action, independent from the serverless environment where they will run. Remember: We don’t want ten pages of code for one action—we want something readable and maintainable.

Talking about staying independent, I’ve used the Promise API. OpenWhisk also has a whisk global object acting as the interface between your code and the activation environment. You may still encounter examples using whisk.async or whisk.done but the Promise approach feels like the right one to go with moving forward.

Nock, nock. Who’s there? The HTTP mocking library!

At some point, your action will likely depend on one or more external services or systems—the less dependencies, the easiest to test. There are several patterns you can follow there. One is about mocking these external services.

I’ve used the nock library for this purpose. It is easy to integrate. It works by intercepting calls made to the Node.js http module. Once enabled, you can return arbitrary response body, status code, headers, and more for specific URL patterns. In the example above, the call to Cloudant is intercepted and a response is generated.

To check the behavior of the action when Cloudant fails, I can fake a 500 status code:

  it('handles Cloudant errors', (done) => {

      demoGuid: 'MyGUID',
      'services.cloudant.url': 'http://cloudant',
      'services.cloudant.database': 'recommendations'
    }).catch((result) => {
      assert.equal(false, result.ok, 'an error should have been detected');

As I write this, I can already hear some commenting that we are no longer unit testing—I’m testing integration with another system. Maybe. But if this helps me gain confidence when I make changes to my action code along the way, I see no reason why I would not do this.

One may argue that if Cloudant changes the structure of its API calls, my tests would still pass but fail once deployed. This is true. The answer to this is simple: unit tests are only part of a larger picture. Of course, you will want to set up higher level testing with integration tests and system tests to achieve greater coverage. These tests would use real external services, not more mocking.

We will be victorious

The good news is that although serverless is a new beast we need to master, we are not coming empty-handed. Our existing skills, know-how and tools still work in this environment. Obviously new tools and techniques will appear as the serverless deployments get more popular, but we are in good company. And with serverless, there is less (read: zero) time spent on the config of runtimes or servers—therefore more time to write tests.

Unit tests are also better to be executed with every commit if you want them to be useful. Using the continuous integration approach described in a previous article, I could easily integrate the unit tests into our build pipeline so that the build would fail if one unit test does not complete successfully.

I encourage you to take a deeper look at the project on GitHub. It has the source code for all the actions listed in the architecture diagram above, along with their tests.

If you have feedback, suggestions, or questions about this post, please reach out to me on Twitter @L2FProd.

Offering Manager - IBM Cloud

More How-tos stories
May 1, 2019

Two Tutorials: Plan, Create, and Update Deployment Environments with Terraform

Multiple environments are pretty common in a project when building a solution. They support the different phases of the development cycle and the slight differences between the environments, like capacity, networking, credentials, and log verbosity. These two tutorials will show you how to manage the environments with Terraform.

Continue reading

April 29, 2019

Transforming Customer Experiences with AI Services (Part 1)

This is an experience from a recent customer engagement on transcribing customer conversations using IBM Watson AI services.

Continue reading

April 26, 2019

Analyze Logs and Monitor the Health of a Kubernetes Application with LogDNA and Sysdig

This post is an excerpt from a tutorial that shows how the IBM Log Analysis with LogDNA service can be used to configure and access logs of a Kubernetes application that is deployed on IBM Cloud.

Continue reading