Use Clojure to write OpenWhisk actions, Part 3

Improve your OpenWhisk Clojure applications

Learn how by developing an inventory control system

Ori Pomerantz
Published on October 30, 2017

Content series:

This content is part # of # in the series: Use Clojure to write OpenWhisk actions, Part 3

https://www.ibm.com/developerworks/library/?series_title_by=**auto**

Stay tuned for additional content in this series.

This content is part of the series:Use Clojure to write OpenWhisk actions, Part 3

Stay tuned for additional content in this series.

In the previous two tutorials, you learned how to write a basic OpenWhisk application using Clojure—a functional programming language based on Lisp—to create actions for OpenWhisk applications. In this tutorial, I'll wrap up the series by showing you how to improve any such application. First, you'll learn how to support arguments that include double quotes. Then I'll show you how to use a permanent database (Cloudant) instead of a variable to store the information.

What you'll need to build your application

This tutorial builds upon information from the first two tutorials in the "Use Clojure to write OpenWhisk actions" series, Part 1, Write clear, concise code for OpenWhisk using this Lisp dialect and Part 2, Connect your Clojure OpenWhisk actions into useful sequences, so I recommend reading those first. In addition, you will need:

A basic knowledge of OpenWhisk and JavaScript (Clojure is optional; the tutorial explains what you need when you need it)
A free Bluemix account (Sign up here)

Run the app Get the code

“Functional programming encourages you to isolate side effects and separate them from the business logic. This results in applications that are more modular, easier to test, and easier to debug.”

Parameters that include quotes

In Part 1, I introduced the main.js JavaScript:

// Get a Clojure environment
var cljs = require('clojurescript-nodejs');


// Evaluate the action code
cljs.evalfile(__dirname + "/action.cljs");


// The main function, the one called when the action is invoked
var main = function(params) {

  var clojure = "(ns action.core)\n ";

  var paramsString = JSON.stringify(params);
  paramsString = paramsString.replace(/"/g, '\\"');

  clojure += '(clj->js (cljsMain (js* "' + paramsString + '")))';

  var retVal = cljs.eval(clojure);
  return retVal;
};

exports.main = main;

That script uses a very simplistic solution to getting the parameters to Clojure:

var paramsString = JSON.stringify(params);
paramsString = paramsString.replace(/"/g, '\\"'); 
…
'(js* "' + paramsString + '")))';

This solution can produce a string such as {\"num\": 5, \"str\": \"whatever\"}. The double backslash (\\) is translated into a single backslash (which is the escape character). The Clojure code that results is (js* " {\"num\": 5, \"str\": \"whatever\"}"). Because js* evaluates the string parameter it gets as JavaScript, this brings us back to the original parameters, {"num": 5, "str": "whatever"}. The problem is that if one of the string parameters already contains a double quote ("), it will be treated exactly the same way as the quotes, leading to an expression such as {"num": 5, "str": "what"ever"} and syntax errors. In theory, you could solve this problem by using the js/<var name> syntax to access a variable with the parameters, but for some reason this doesn’t work in OpenWhisk.

Here in Part 3, I introduce the fixHash function, which iterates over the parameters (including any nested data structures) and looks for strings. In a string, it replaces all double quotes with \\x22. The first backslash escapes the second backslash, so the real value is \x22. This value eventually gets translated into ASCII character 0x22 (decimal 34, which is double quotes), but that happens at a later stage so the replace method does not modify these characters.

// Fix a hash table so it won't have internal double quotes
var fixHash = function(hash) {
	if (typeof hash === "object") {
		if (hash === null) return null;
		if (hash instanceof Array) {
			for (var i=0; i<hash.length; i++)
				hash[i] = fixHash(hash[i]);
			return hash;
		}
		
		var keys = Object.keys(hash) 
		for (var i=0; i<keys.length; i++)
			hash[keys[i]] = fixHash(hash[keys[i]]);
		return hash;
	}

	if (typeof hash === "string") {
		return hash.replace(/"/g, '\\x22');
	}

	return hash;
		
};

Save to a database

An inventory management system that resets to the initial value when not in use for a few minutes isn't very useful. Therefore, your next step is to set up an object storage instance to store the database values (the dbase variable in the inventory_dbase action):

In the Bluemix console, click on the Menu icon and go to Services > Data & Analytics.
Click Create Data & Analytics service and select Cloudant NoSQL DB.
Name the service "OpenWhisk-Inventory-Storage" and click Create.
When the service is created, open it and click Service credentials > New credential.
Name the new credential "Inventory-App" and click Add.
Click View credentials and copy the credential into a text file.
Select Manage > Launch.
Click on the database icon and Create Database.
Name the database "openwhisk_inventory."
Click the plus icon in the All Documents row and select New Doc.
Copy the contents of dbase.json into the text area and click Create Document.

There are two ways to combine the Cloudant database with the application. You can either add Cloudant actions to the sequences, or modify the inventory_dbase action. I chose the second solution, because it lets me change a single action (because all the database work is centralized there).

Add the npm library for Cloudant to the dependencies in package.json, and update the inventory_dbase action's action.cljs file:

(ns action.core)

(def cloudant-fun (js/require "cloudant"))

(def cloudant (cloudant-fun "url goes here"))

(def mydb (.use (aget cloudant "db") "openwhisk_inventory"))



; Process an action with its parameters and the existing database
; return the result of the action
(defn processDB [action dbase data]
  (case action
    "getAll" {"data" dbase}

    "getAvailable" {"data" (into {} (filter #(> (nth % 1) 0) dbase))}

    "processCorrection" (do
      (def dbaseNew (into dbase data))
      {"data" dbaseNew}
    )

    "processPurchase" (do
      (def dbaseNew (merge-with #(- %1 %2) dbase data))
      {"data" dbaseNew}
    )

    "processReorder" (do
      (def dbaseNew (merge-with #(+ (- %1 0) (- %2 0)) dbase data))
      {"data" dbaseNew}
    )

    {"error" "Unknown action"}
  )   ;  end of case
)   ; end of processDB



(defn cljsMain [params] (
    let [
      cljParams (js->clj params)
      action (get cljParams "action")
      data (get cljParams "data")
      updateNeeded (or (= action "processReorder")
                       (= action "processPurchase")
                       (= action "processCorrection"))
    ]

    ; Because promise-resolve is here, it can reference
    ; action
    (defn promise-resolve [resolve param] (let
      [
        dbaseJS (aget param "dbase")
        dbaseOld (js->clj dbaseJS)
        result (processDB action dbaseOld data)
        rev (aget param "_rev")
      ]
        (if updateNeeded
          (.insert mydb (clj->js {"dbase" (get result "data"),
                                  "_id" "dbase",
                                  "_rev" rev})
            #(do (prn result) (prn (get result "data")) (resolve (clj->js result)))
          )
          (resolve (clj->js result))
        )
      )   ; end of let
    )   ; end of defn promise-resolve


    (defn promise-func [resolve reject]
      (.get mydb "dbase" #(promise-resolve resolve %2))
    )

    (js/Promise. promise-func)
  )   ; end of let
)    ; end of cljsMain

Let's look at some of the more interesting parts of action.cljs.

You need to get the database. In JavaScript, you might code it this way:

var cloudant_fun = require("cloudant ");
var cloudant = cloudant_fun(<<<URL>>>);
var mydb = cloudant.db.use("openwhisk_inventory ");

The Clojure version of the same code is similar, with a few differences. First, require is a JavaScript function. To access it, you need to qualify it with the js namespace (line 3 in the listing in step 12 above):

(def cloudant-fun (js/require "cloudant"))

The next line (line 5) is pretty standard. The URL is a URL parameter from the credentials for the database:

(def cloudant (cloudant-fun <<URL GOES HERE>>))

To get a member from a JavaScript object, use aget. To use a method of an object, you can use (.<method> <object> <other parameters>). See line 7:

(def mydb (.use (aget cloudant "db") "openwhisk_inventory"))

Reading from the database and writing to it are both asynchronous actions. This means that you cannot simply run them and return the result to the caller (the OpenWhisk system). Instead, you need to return a Promise object. This constructor accepts a single parameter—a function to be called to start the process whose result is needed. The syntax to call the constructor for a JavaScript object from Clojure is (js/<object name>. <parameters>). See line 75:

    (js/Promise. promise-func)

The function that is provided to the Promise object's constructor is promise-func. It receives two arguments. One is the function to call in case of success (one argument, the result of the action). The other is the function to call in the case of failure (also one argument, the error object). In this case, the function gets the dbase document and then calls promise-resolve with the success function and the document. The first parameter to the anonymous function (#(promise-resolve resolve %2)) is the error, if any. Because this is a demonstration program, we ignore errors for the sake of simplicity. See lines 71-73:

    (defn promise-func [resolve reject]
      (.get mydb "dbase" #(promise-resolve resolve %2))
    )

Both promise-func and promise-resolve are defined inside cljsMain. The reason is that promise-resolve needs the value of the action parameter. By defining these functions inside cljsMain, you can use just that local variable without having to drag it along for the entire calling chain. See lines 52-55:

    (defn promise-resolve [resolve param] (let
      [
        dbaseJS (aget param "dbase")
        dbaseOld (js->clj dbaseJS)

The function that either gets the data or modifies it is processDB. This function encapsulates the functionality explained in Part 1. See line 56:

        result (processDB action dbaseOld data)

The revision (_rev) is necessary to update the database because of the algorithm that Cloudant uses to ensure state consistency. When you read a document, you get the current revision (_rev). When you write the updated version, you have to provide Cloudant with the revision you are updating. In the meantime, if another process has updated the document, the versions won't match and the update will fail. See line 57:

        rev (aget param "_rev")

If the data has been modified, update Cloudant. See line 59:

        (if updateNeeded

Provide the database with the new data, the document name, and the revision you are updating. See lines 60-62:

          (.insert mydb (clj->js {"dbase" (get result "data"),
                                  "_id" "dbase",
                                  "_rev" rev})

When the update is done (we just assume it is successful; this is an educational sample, not production code), run the function below. Notice the mechanism to add debugging printouts. Use do to evaluate several expressions, have any number of prn function calls to print the information you need, and put the expression you actually want last.

In this case, you call the resolve function that you got through the Promise object. Because this function is in JavaScript, it needs to receive a JavaScript object, not a Clojure one, so you use clj->js. See lines 63-64:

            #(do (prn result) (prn (get result "data")) (resolve (clj->js result)))
          )

If there is no need to update Cloudant, just run the resolve function. See lines 65-66:

          (resolve (clj->js result))
        )

Compile the Clojure code

Currently, we send the Clojure code to OpenWhisk and compile it there whenever the Node.js applications restarts. Another option is to compile the Clojure to JavaScript locally once and send the already compiled version. If you are interested, you can see how it works here. However, this method does not improve performance significantly.

As you can see in the screen capture below, even with compiled ClojureScript code, the first invocation of the action still takes a lot more time than subsequent ones.

Screen capture showing first invocation of action

View image at full size

The reason for the slowness is that the actual compilation is not very resource intensive. The creation of the Clojure environment is the resource-intensive part of using Clojure, and that is required even if the Clojure code itself is compiled into JavaScript. The JavaScript that's produced by the compiler uses some heavy libraries.

Conclusion

This concludes the series on OpenWhisk actions in Clojure. I hope I have shown you some of the advantages to using functional programming for functions as a service (FaaS). At the end of the day, nearly every application needs to use side effects at some point, but functional programming encourages you to isolate those side effects and separate them from the business logic. This results in applications that are more modular, easier to test, and easier to debug. By separating the application logic into actions and sequences, it is easy to make the large-scale structure of the application clearer, and write unit tests to run as REST calls.