Use Clojure to write OpenWhisk actions, Part 1
Write clear, concise code for OpenWhisk using this Lisp dialect
Learn how by developing an inventory control system
Content series:
This content is part # of # in the series: Use Clojure to write OpenWhisk actions, Part 1
This content is part of the series:Use Clojure to write OpenWhisk actions, Part 1
Stay tuned for additional content in this series.
Interested in functional programming? How about function as a service (FaaS)? In this tutorial, you learn to combine these two by writing OpenWhisk actions in Clojure. Such actions can be clearer and more concise than those written in JavaScript. Also, functional programming is a better paradigm for FaaS because it encourages programming without reliance on side effects.
This is the first in a series of three tutorials that illustrate Clojure and OpenWhisk through the development of an inventory control system. Here in Part 1, you learn how to use Clojure to write OpenWhisk actions using the Node.js runtime and the ClojureScript package. Part 2 teaches you how to use OpenWhisk sequences to combine actions into useful chunks that do the work of an application. And Part 3 shows you how to interface with an external database, and how to use logging to debug your own Clojure in OpenWhisk applications.
What you'll need to build your application
- Basic knowledge of OpenWhisk and JavaScript (Clojure is optional, the article explains what you need when you need it)
- A Bluemix account (sign up here).
Why do this?
Actually, this is really two separate questions:
- Why write OpenWhisk actions in Clojure, as opposed to the native JavaScript?
- Why use OpenWhisk if you are going to write in Clojure?
Let's take a look at each of these in turn...
Why write OpenWhisk actions in Clojure, as opposed to the native JavaScript?
Clojure is a dialect of Lisp, and provides all the programming advantages of that language (such as immutability and macros). It takes some getting used to, but once you do you can write clear, concise code. For example, this single line takes over 450 words to explain later in this article, and anybody who understands Clojure can comprehend it at a glance:
"getAvailable" {"data" (into {} (filter #(> (nth % 1) 0) dbase))}
Why use OpenWhisk if you are going to write in Clojure?
FaaS platforms, such as OpenWhisk, make it easy to build highly modular systems that communicate only through well-defined interfaces. This makes it easy to develop applications that are modular, without any dependencies on side effects. Also, FaaS requires fewer resources and is therefore cheaper than having a constantly running application.
The development toolchain
IBM does not officially recommend Clojure, and OpenWhisk does not have a Clojure runtime. The way we will run Clojure on OpenWhisk is by using the ClojureScript package, which compiles Clojure code to JavaScript. The JavaScript can then be executed by the Node.js runtime.
The most common way to code an action using the Node.js runtime is to put everything into a single file, with a main function that receives the parameters and returns the result. This method is simple, but your code is limited to using whatever npm libraries OpenWhisk already has.
Alternatively, you can write a more complete Node.js program with a
package.json file, put it into a zip file, and then upload it. This allows
you to use other libraries, such as clojurescript-nodejs
. For
more details, read "Creating Zipped Actions in OpenWhisk" by Raymond Camden.
The Windows Store includes a Linux subsystem that you can run right from Windows. Personally, I prefer to install the toolchain on Linux—that way, I can do it directly from my Windows laptop. The commands below are issued in that environment.
- Install npm (this can be a time-consuming process because it requires
a lot of other packages):
sudo apt-get update sudo apt-get install npm
- Create a package.json file with this content (available on GitHub):
{ "name": "openwhisk-clojure", "version": "1.0.0", "main": "main.js", "dependencies": { "clojurescript-nodejs": "0.0.8" } }
Note: The current package version is 0.0.8. By specifying the version in the package.json file, you ensure that the application will not break if in the future a version is released that isn’t backwards compatible. - Create a main.js file (available on GitHub):
// Get a Clojure environment var cljs = require('clojurescript-nodejs'); // Evaluate the action code cljs.evalfile(__dirname + "/action.cljs"); // The main function, the one called when the action is invoked var main = function(params) { var clojure = "(ns action.core)\n "; var paramsString = JSON.stringify(params); paramsString = paramsString.replace(/"/g, '\\"'); clojure += '(clj->js (cljsMain (js* "' + paramsString + '")))'; var retVal = cljs.eval(clojure); return retVal; }; exports.main = main;
- Create an action.cljs file (available on GitHub):
(ns action.core) (defn cljsMain [params] {:a 2 :b 3 :params params} )
- Run this command to install the dependencies:
npm install
- Install the zip program.
sudo apt-get install zip
- Zip the files necessary for the action.
zip -r action.zip package.json main.js action.cljs node_modules
- Download the wsk executable for Linux (this link is for the 64 bit
version). Put it in a directory that is in the path, for example
clojurescrip/usr/local/bin
.sudo mv wsk /usr/local/bin
- Get your authentication key and run the
wsk
command to log on.wsk property set --apihost openwhisk.ng.bluemix.net --auth <your key here>
- Upload the action (in this case, name it
test
).wsk action create test action.zip --kind nodejs:6
- Go to the Bluemix OpenWhisk UI, click Develop on the
left sidebar, and run the action
test
. The response should be similar to the following screen capture:
Note: If you look in the logs for the action, it will show that you're using an undeclared variable. You can safely ignore that warning message.
How does it work?
I have written before on how to integrate Clojure and Node.js, so the explanation here will be somewhat abbreviated. If you want more details, you can always find them there.
Looking at the stub, main.js, it starts with code that creates a ClojureScript (Clojure that is translated into JavaScript rather than Java) environment and then evaluates the action.cljs file.
// Get a Clojure environment var cljs = require('clojurescript-nodejs'); // Evaluate the action code cljs.evalfile(__dirname + "/action.cljs");
This approach is simple, and while it requires the Clojure to be recompiled
every time the action is restarted, that is not as bad as it sounds. The
initialization code (the code that is not in main
or called
by the code in main
) is executed once and then the results
are cached by OpenWhisk. So the Clojure only gets recompiled when the
action isn't invoked for a long period.
Next is the main
function. It is called with the parameters in
a JavaScript hash table.
// The main function, the one called when the action is invoked var main = function(params) {
We start to create the Clojure code by declaring ourselves part of the
action.core
namespace.
var clojure = "(ns action.core)\n ";
Getting the parameters into Clojure is a bit complicated. When simpler
solutions failed, I turned to this one, which encodes the parameters as a
JavaScript Object Notation (JSON) string. JSON can be evaluated as a
JavaScript expression, which can be evaluated in ClojureScript using the
syntax (js* <JavaScript expression>)
. However, the
JavaScript expression is a string, and strings in
Clojure are enclosed in double quotes ("), the same character that
JSON.stringify
uses. Therefore, the next line makes sure the
double quotes in the parameter string are escaped. Note that this
simplistic solution fails when the parameter values include a double
quote; I plan to show a better solution in the third article in this
series.
var paramsString = JSON.stringify(params); paramsString = paramsString.replace(/"/g, '\\"');
This line adds the code that actually calls the action in Clojure. In
Clojure (and its ancestor, Lisp), a function call is not expressed as the
usual function(param1, param2, ...)
, but as
(function param1 param2 ...)
. Going from the innermost
parenthesis to the outermost, this code first takes the parameter string
and interprets it as a JavaScript expression. Then, it calls the function
cljsMain
with that value. The output of
cljsMain
, a Clojure hash table, is then converted to a
JavaScript hash table using clj->js
.
clojure += '(clj->js (cljsMain (js* "' + paramsString + '")))';
Finally, call the Clojure code and return the return value:
var retVal = cljs.eval(clojure); return retVal; };
This line exports the main
function, so it will be available
to the runtime.
exports.main = main;
The action itself in action.cljs is even simpler. The first line declares
the name space, action.core
. Clojure originated as a Java
Virtual Machine language, and the namespace has some of the functions of
the class name in Java.
(ns action.core)
This code defines the cljsMain
function. In general, Clojure
functions are defined using
(defn <function name> [<parameters>] <expression>)
.
The expression is usually a function call, but it does not have to be.
Here, it is a literal expression. Hash tables in Clojure are enclosed by curly
brackets ({}
). The syntax is
{<key1> <value1> <key2> <value2> ...}
.
The keys in this case are keywords, words that start with a colon
(:
), which in Clojure means they cannot be symbols for
anything else. The values in this hash table are two numbers and the
parameters passed to the action.
(defn cljsMain [params] {:a 2 :b 3 :params params} )
Inventory control system
The sample application for this article is an inventory control system. It has two front ends—one is a point of sale that reduces the inventory, and the other is a reordering system that lets managers purchase replacement items or correct inventory numbers.
“Database” action
To abstract the database, create one action that handles all the database interactions. Based on the parameters, this action needs to perform one of the following actions:
- getAvailable—Get the list of available items (those that you have in stock), and how many you have of each.
- getAll—Get the list of all items, including those that are out of stock, for reordering.
- processPurchase—Get a list of items and how many of each were purchased, and deduct them from the inventory.
- processReorder—Get a list of reordered items and amounts, and add it to the inventory.
- processCorrection—Get a list of items and the correct amounts (after the stock is physically counted). This amount may be more or less than the amount currently in the database.
For now, the database is going to be a hash table, with the item names as keys and the amount in stock as the value. Note that this value is going to be reset every time the process for the action is restarted.
- In a new directory (for example, …/inventory/dbase_mockup) create the same three files you created for the test action: package.json, main.js, and action.cljs. The first two have the same content that they did in the test action. You can find the third, action.cljs, in GitHub.
- Run this command to install the dependencies:
npm install
- Zip the action:
zip -r action.zip package.json main.js action.cljs node_modules
- Upload the action:
wsk action create inventory_dbase action.zip --kind nodejs:6
- Run the action with test inputs to see what happens:
Input Expected result { "action": "getAll" }
{ "data": { "T-shirt L": 50, "T-shirt XS": 0, "T-shirt M": 0, "T-shirt S": 12, "T-shirt XL": 10 } }
{ "action": "getAvailable" }
{ "data": { "T-shirt L": 50, "T-shirt S": 12, "T-shirt XL": 10 } }
{ "action": "processCorrection", "data": {"T-shirt L": 10, "Hat": 15} }
{ "data": { "T-shirt L": 10, "Hat": 15, "T-shirt XS": 0, "T-shirt M": 0, "T-shirt S": 12, "T-shirt XL": 10 }
{ "action": "processPurchase", "data": { "T-shirt L": 5, "T-shirt S": 2 } }
{ "data": { "T-shirt L": 5, "Hat": 15, "T-shirt XS": 0, "T-shirt M": 0, "T-shirt S": 10, "T-shirt XL": 10 } }
{ "action": "processReorder", "data": { "T-shirt L": 20, "T-shirt M": 30 } }
{ "data": { "T-shirt L": 25, "Hat": 15, "T-shirt XS": 0, "T-shirt M": 30, "T-shirt S": 10, "T-shirt XL": 10 } }
How does it work?
This section introduces a number of Clojure concepts. It is recommended that you read it with a browser tab opened to a Clojure command line (called REPL, for "read, evaluate, and print loop") to learn by doing.
The first line of action.cljs defines the namespace:
(ns action.core)
Next, we use the def
command to define dbase
to
be a hash table. The syntax is somewhat similar to the syntax in
JavaScript, but there are several important differences:
- There is no colon (
:
) between the key and the value. - You can use a comma (
,
) as a separator between different key-value pairs ({"a" 1, "b" 2, "c" 3}
). However, you can also omit the separator without changing the expression's value (so{"a" 1 "b" 2}
would be the same as{"a" 1, "b" 2}
). - You don’t see it here, but the key does not have to be a string; it
can be any legitimate value:
(def dbase { "T-shirt XL" 10 "T-shirt L" 50 "T-shirt M" 0 "T-shirt S" 12 "T-shirt XS" 0 } )
Then, defn
is used to define the function
cljsMain
. It takes a single parameter, a hash table with the
parameters. Because of the way main.js is written, this is a JavaScript
hash table, not a Clojure one.
(defn cljsMain [params] (
The next line uses the let
function. This function gets a
vector—essentially a list enclosed by square brackets
([]
)—and an expression. The vector has identifiers
followed by the value to assign to them for the duration of the
let
expression. Using let
allows you to program
in a format that is close to imperative programming. The code inside the
vector could be written in JavaScript as:
var cljParams = js→clj(params); var action = get(cljParams, "action"); var data = get(cljParams, "data");
This line starts the code in Clojure:
let [
As I mentioned above, the value in params
is a JavaScript hash
table. The js->cljs
function translates it to a Clojure
hash table (the reverse of cljs->js
used in main.js).
cljParams (js->clj params)
The other two symbols, action
and data
, get the
values of specific parameters. One way to get the value in a hash table is
the function
(get
<hash table> <value>)
. Not
all actions have a data
parameter, but that’s OK—we’ll
just get nil
in those cases, not an error condition.
To see this in action, run the following code on the REPL
website (get {:a 1 :b 2 :c 3} :b)
. Remember, words that
start with a colon are keywords that cannot be used as symbols, so there
is no need to treat them as strings.
action (get cljParams "action") data (get cljParams "data") ]
The function case
acts in the same way as
switch...case
statements in JavaScript (which inherited them
from C, C++, and Java).
(case action
The "getAll"
action is the simplest, just return the database
under the parameter "data"
.
"getAll" {"data" dbase}
The next action looks for available items, those you have in stock. The expression that implements this is not particularly complicated, but it uses several techniques that are specific to functional programming.
In imperative programming, you tell the computer what to do. In functional programming, you tell the computer what you want and let the computer figure out how to do it. In this case, you want the computer to give you all of the items where the number of items is above 0.
To do this, you use the filter
function. This function
receives a function and a list, and returns only those items for which the
parameter function returns a true value (most values are true). When you
give filter
a hash table, it acts as if it is a list of
ordered pairs, each consisting of a key and its value.
The form #(<function>)
defines a function (without
giving it a name, so it is an anonymous function). In that function
definition, you refer to the function’s sole parameter, or the first
parameter if there are several, as a percent (%
or
%1
). You can refer to other parameters as %2
,
%3
, etc. To get a value from a list or a vector, you can use
the nth
function. This function counts from 0, so the first
value in the list is (nth [<list>] 0)
, the second is
(nth [<list>] 1)
, etc.
Run on the REPL
website(nth [:a :b :c :d] 2)
to see how nth
works. To see an anonymous function in action, run
(#(+ 3 %) 3)
. The anonymous function adds three to whatever
value it gets, so the result is 3 + 3, or 6.
The function #(> (nth % 1) 0)
finds the second value in the
parameter and checks if it is higher than 0. Because of the way the
filter
works with hash tables, that would always be the
value, the number of items. For your purposes here, you only care about
the cases where that number is positive.
At this point, the result is a list of vectors, each with two values:
product name and the amount in stock. However, the desired output is a
hash table. To add values formatted in this manner to a hash table, use
the into
function. The first argument for this function is
the initial hash table to which you add values, in this case the empty
one.
To follow along on the REPL
website, run
(filter #(= (nth % 1) 1) {:a 1 :b
0 :c 1 :d 2})
to see the list. Then, run
(into {} (filter #(= (nth % 1) 1)
{:a 1 :b 0 :c 1 :d 2}))
to see the list in a hash table.
"getAvailable" {"data" (into {} (filter #(> (nth % 1) 0) dbase))}
The three other actions modify the database. However, I want them to return
the new database. To do that, you use the do
function. This
function gets a number of expressions, evaluates them, and returns the
last one. This allows for expressions that have side effects, such as
assigning a new meaning to the dbase
symbol.
Processing a correction is easy. Because the corrected values replace the
existing ones, you can use the into
function. It acts as you
would expect, replacing values when the keys are the same.
"processCorrection" (do (def dbase (into dbase data)) {"data" dbase} )
Processing purchases and reorders is more difficult, because it depends on
the values in both the old value in dbase
and the new value
in data
. Luckily, Clojure provides you with a function called
merge-with
, which receives a function and two hash maps. If a
key only appears in one hash, that value is used. If a key appears in both
maps, it runs the function and uses that value.
To follow along on the REPL
website, run
(merge-with #(- %1 %2) {:a 1 :b 2 :c 3} {:b 3 :c 2 :d 4})
.
"processPurchase" (do (def dbase (merge-with #(- %1 %2) dbase data)) {"data" dbase} ) "processReorder" (do (def dbase (merge-with #(+ %1 %2) dbase data)) {"data" dbase} )
After all the value and expression pairs, you can put a default value. In this case, it is an error message.
{"error" "Unknown action"} ) ) )
Conclusion
In this tutorial, you learned how to write a single action in Clojure, the mock database. If you were writing a single-page application, that might be enough. However, to write an entire application in Clojure on OpenWhisk requires other actions that transform the JSON that is the normal output of an action to HTML, and transform HTTP POST requests with new information to JSON. This is the topic of the next tutorial in this series.