Understanding and using Docker actions in IBM Bluemix OpenWhisk
By: Rodric Rabbah
Understanding and using Docker actions in IBM Bluemix OpenWhisk
The latest Apache OpenWhisk offering on IBM Bluemix has several new enhancements, including performance tweaks for running actions as Docker containers.
As a previous post explains, OpenWhisk supports Docker actions in OpenWhisk to execute binaries on demand without provisioning virtual machines. Docker actions are best suited to cases where it is difficult to refactor an application—for example, a monolithic C or Java application with third-party dependencies—into a small set of functions using one or more of the supported OpenWhisk runtimes and languages.
Getting around limitations on Docker actions with OpenWhisk
There are two limitations in using Docker actions:
Latency
When it receives the request to activate a Docker action, OpenWhisk pulls the Docker image from Docker Hub. The size of the image and the network bandwidth between the OpenWhisk host and Docker Hub may introduce latency.
Security
Posting Docker images to a central public hub may not be an option for proprietary code.
A recent enhancement to OpenWhisk eliminates these limitations. Now a Docker action can receive—as initialization—a zip file containing the executable to run. The zip file may contain a shell script or binary called exec, and it may even include supporting files.
All action and no pull
OpenWhisk provides a base image for Docker actions called openwhisk/dockerskeleton. This image is based on Alpine Linux 3.4 and includes Python v2.7.12 but not much else. (For reference, the image is based on python:2.7.12-alpine).
From this base image, you can create and run an OpenWhisk action:
> wsk action create skeleton --docker openwhisk/dockerskeleton
ok: created action skeleton
> wsk action invoke skeleton --blocking --result
{ "error": "This is a stub action. Replace it with custom logic." }
The image contains a stub executable, which typically you would replace with a binary when creating a custom Docker image.
Instead, without creating a new container image, let’s create a new executable to run instead of the default stub.
> echo '#!/bin/bash
echo "{ \"hello\": \"ran without a docker pull!\" }"' > exec
We created a file called exec that includes a simple shell script. In order to run this action inside the container, we change the file type to be executable.
> chmod +x ./exec
> ./exec
{ "hello": "ran without a docker pull!" }
Preparing to replace the executable stub in the Docker skeleton, we zip our new exec file and use the compressed archive to create the Docker action.
> zip exec.zip exec > wsk action create noPull exec.zip --docker ok: created action noPull
The command created an OpenWhisk Docker action using the openwhisk/dockerskeleton image and exec.zip as an initializer. The OpenWhisk CLI restricts use of the .zip attachment to Docker actions that use openwhisk/dockerskeleton as an image.
Advanced tip: While it is possible to create a Docker action with a zip initializer from any base image, the CLI currently restricts using this feature to just the openwhisk/dockerskeleton image. This is because the Docker action interface is not officially documented and subject to change. It is possible to circumvent this CLI restriction by using the REST API directly; this permits using the feature with custom Docker actions that reverse-engineer the OpenWhisk action protocol, for example.
We are ready to run the new action and observe the result:
> wsk action invoke noPull --blocking --result
{ "hello": "ran without a docker pull!" }
And there it is: a new Docker action without a new Docker container.
The executable need not be a Bash script, of course. There are only two requirements for what you run as an executable:
-
That the zip file must contain a top-level executable file called
exec. -
That any binary executable is compatible with Alpine Linux 3.4.
Hello 62696e6172790d0a
One way to generate a binary to inject into the Docker skeleton is to compile it directly inside the openwhisk/dockerskeleton image. You can do this by starting a local container from this image and mounting some local directory into the host container.
> docker run -it -v "$HOME:/owexec" openwhisk/dockerskeleton bash
Now, inside the container, let’s create an example C file to run as an OpenWhisk action:
> cd /owexec
> echo '#include <stdio.h>
int main(int argc, char *argv[]) {
printf("{ \"hello\": \"ran a binary without a docker pull!\" }");
}'> example.c
It is necessary to install gcc and some dependencies to compile the C code inside the container. Notice the use of the -static flag here to statically link the binary:
> apk add --update gcc libc-dev > gcc -static -o exec example.c
That’s it as far the binary is concerned. Exit the container and return to the local host where the files example.c and exec should now exist.
> zip binary.zip exec
> wsk action create binary binary.zip --docker
> wsk action invoke binary --blocking --result
{ "hello": "ran a binary without a docker pull!" }
You can compare the performance of this action to one created from a custom Docker image. A comparable binary image is provided on Docker Hub called openwhisk/example.
> wsk action create dockerPull --docker openwhisk/example
> wsk action invoke dockerPull --blocking --result
{ "args": {}, "msg": "Hello from arbitrary C program!" }
Did you notice the latency differential? (It is approximately half a second.) While subsequently invoking the same action will get the benefit of the cached Docker image, the initial cold-start latency can be significant.
How do you spell that?
You can use this feature in even more powerful ways—to install custom dependencies on the fly before running actions, for example.
Let’s revisit the spell-check Docker action. This action runs the GNU aspell spell-checker to find misspelled words in a file. Whereas previously we would push a Docker container image to Docker Hub, here we will run the same action without a custom container.
Since this action depends on aspell, a third-party dependency that does not exist in the OpenWhisk Docker skeleton, we must install it inside the container running the action. (This was the case in the previous post.)
An alternative is to compile and statically link the dependencies into a native binary that can be injected into the container, but this can be time consuming and difficult to do for large applications.
A third option is to dynamically install the dependencies as needed. The Bash script below implements the aspell action taken verbatim from the previous article on Docker actions and modifies it in two ways:
-
The base64
--decodeoptionis replaced with-dto match the version installed in the new base image. -
Three lines are added to install the third party dependencies when necessary (see
NEWcomment below). Since repeated action activations may reuse the same container, the action can skip installingaspellwhen it was already installed by a previous activation.> cat exec #!/bin/bash # # This script expects one argument, a String representation of a JSON # object with a single attribute called b64. The script decodes # the b64 value to a file, and then pipes it to aspell to check spelling. # Finally it returns the result in a JSON object with an attribute called # result. # FILE=/tmp/output.txt # NEW: The next three lines install third party dependencies when necessary. if [ ! -f /usr/bin/aspell ]; then apk add --update aspell aspell-en fi # Parse the JSON input ($1) to extract the value of the 'b64' attribute, # then decode it from base64 format, and dump the result to a file. echo $1 | sed -e 's/b64.://g' \ | tr -d '"' | tr -d ' ' | tr -d '}' | tr -d '{' \ | base64 -d >& $FILE # Pipe the input file to aspell, and then format the result on one line with # spaces as delimiters RESULT=`cat $FILE | aspell list | tr '\n' ' ' ` # Return a JSON object with a single attribute 'result' echo "{ \"result\": \"$RESULT\" }"
To run this code as an OpenWhisk action, repeat the earlier steps.
> zip aspell.zip exec
> wsk action create aspell aspell.zip --docker
> wsk action invoke aspell --blocking --result --param b64 \
`echo cat dot elephent lion mooose bug | base64 -`
{ "result": "elephent mooose " }
You’ll notice a delay with the first activation. This is the time spent installing the third-party dependencies. When you run the action again, you’ll notice it is much faster.
In short, you are trading off a Docker pull of your image from the public registry for a dynamic installation of dependencies. Trade off wisely!
Ready, set, go!
There is another way to use this Docker action feature of OpenWhisk—when your favorite coding language does not yet have first-class OpenWhisk support.
In this example, we run an action using the Go programming language. It illustrates a general pattern for creating a stub to install the required dependencies, passing the arguments to the action, and working with the JSON input inside the action.
First, we create the generic exec wrapper for installing Go and invoking the actual action.
#!/bin/bash if [ ! -f /usr/bin/go ]; then apk add --update go fi /usr/bin/go run /action/exec.go "$@"
Next, we store the Go action in the file exec.go.
package main
import "encoding/json"
import "fmt"
import "os"
func main() {
// native actions receive one argument, the JSON object as a string
arg := os.Args[1]
// unmarshal the string to a JSON object
var obj map[string]interface{}
json.Unmarshal([]byte(arg), &obj)
name, ok := obj["name"].(string)
if !ok { name = "Stranger" }
msg := map[string]string{"msg": ("Hello, " + name + "!")}
res, _ := json.Marshal(msg)
fmt.Println(string(res))
}
Finally, we zip and create the OpenWhisk action.
> zip golang.zip exec exec.go
> wsk action create go golang.zip --docker
> wsk action invoke go --blocking --result --param name OpenWhisk
{ "msg": "Hello, OpenWhisk!" }
Repeat for profit. Enjoy.
