Rapidly develop Internet of Things apps with Docker Containers
Develop, test, deploy, and update IoT apps using container-based virtualization
The Internet of Things, the highly connected network of smart devices, such as environmental sensors, medical trackers, home appliances, and industrial devices, is growing rapidly. By 2020, a predicted 20 billion devices will be connected, which is more than twice the number of PCs, smartphones, and tablets combined. Developers are rapidly starting to create applications for the IoT, and using containers can help them in different ways.
Containers are a lightweight approach to virtualization that developers can apply to rapidly develop, test, deploy, and update IoT applications at scale. Many web and mobile app developers make use of hypervisors, such as VirtualBox, to run virtual machines (VMs) that virtualize physical hardware as part of a cross-platform development, testing, and deployment workflow.
Container-based virtualization (sometimes called operating-system-level virtualization) is much more lightweight. Each container runs as an isolated user space instance on top of a shared host operating system kernel. Although the operating system is shared, individual containers have independent virtual network interfaces, independent process spaces, and separate file systems. These containers can be allocated with system resources like RAM by using control groups that implement resource isolation. Compared to hypervisor-based virtualization, where each VM runs its own operating system which only increases its use of system resources, containers use much less disk and memory resources.
Docker is an open platform for container-based virtualization on Linux. Docker makes it fast and easy to build containers and to deploy them just about anywhere: in a private or public cloud, within a local VM, or on physical hardware including IoT devices. IBM Containers are a Bluemix beta feature based on Docker for delivering and deploying containerized applications on the Bluemix platform.
Working with containers, images, and registries
Each Docker container contains one or more running processes. The Docker container is launched from an image that specifies these elements:
- The configuration information for an application to be run inside the container
- Its dependencies, for example, libraries and shared binary files
Docker makes use of a union file system to store images as a series of layers. Layers are cached during the build process, which makes building derivative images fast and efficient.
This layered approach also means that Docker images are small and portable, which makes them easy to share by publishing them to a public or private registry. Docker Hub is the largest registry of Docker image repositories; it lists more than 100,000 repositories and hosts numerous private repositories. If you are using the IBM Bluemix cloud platform, it supports pulling public images from Docker Hub and also provides hosted private image repositories which allow you to share private images within your organization.
If you're using a popular open source framework or service, it's likely that you'll be able to find a pre-built public image in Docker Hub, with many of the images maintained by the open source communities that are associated with those projects. The Docker Hub web interface features a small list of official repositories, which is a curated list of repositories that include images that are tested by the Docker team for known security vulnerabilities.
Avoid using older Docker images, even from reliable sources, as many older images are provided for archival and testing purposes only. Older images are tagged as supported in Docker Hub if they are still maintained. The majority of images in the public registry are community contributed and may not be well documented or maintained. When in doubt, create your own image using one of the official images as a base.
You can create an image by completing these steps:
- Pull a base image from the registry
- Run a series of commands interactively
- Commit the result as a new image
For example, the following list of commands pulls the official Ubuntu version 14.04 image from the Docker Hub, launches a container from the image to run a command that installs Git into the container, and then lists the ID of the container that was created.
Listing 1. Creating an image
$ docker pull ubuntu:14.04 $ docker run ubuntu apt-get install -y git $ docker ps -l
You can save this container (cb23e345fde0) as a new image (named demo/git)
by issuing the following command:
$ docker commit cb23e345fde0 demo/git.
To run Docker commands locally by using the IBM Container Extensions tool,
docker command to
ice --local. For
example, use this command:
$ ice --local pull ubuntu:14.04
If you need to run more than one or two commands to set up your application environment, you'll want to create a Dockerfile. Dockerfiles are text files that specify a list of instructions for building an image. The Dockerfile in the following code listing creates a base image for running an Nginx web server.
Listing 2. Dockerfile that creates a base image for running an Nginx web server
FROM ubuntu:14.04 RUN apt-get -y update && apt-get install -y nginx RUN mkdir -p /var/www/html EXPOSE 80 EXPOSE 443 WORKDIR /etc/nginx CMD ["nginx"]
The first instruction in each Dockerfile is a
instruction, which indicates the base image. Each subsequent instruction
in the Dockerfile is stored as a layer:
RUNinstructions run Linux commands such as
COPYinstructions add application files into the container.
EXPOSEinstructions open ports.
ENVinstructions configure environment variables.
Finally, each Dockerfile contains
CMD instructions to specify how
and where to run the application when the container is launched.
Running Docker containers requires very little overhead, so you can break your application up into a set of services that run in separate containers that can be linked by name when you run them. For example, you might split your application into a container running a Node.js application linked to another container running a Redis key-value store.
Maintaining a consistent IoT development environment using containers
IoT applications target a wide variety of device platforms. During prototyping and early phases of development, IoT projects are often developed using generic microcontroller-based development boards or single-board computers (SBCs) like the Raspberry Pi. The embedded applications that are developed for these devices can later be upgraded to run on custom prototype development boards and then finally on production devices. You might evaluate a range of potential microcontrollers and system-on-a-chip (SoC) devices, with distinct and sometimes incompatible device driver requirements. Each device or device revision might also require different versions and configurations of the development toolkits that are used for flashing, monitoring, and communicating with the device.
Containers can be used to capture a development environment that is known to work for each device revision, and to share this environment among a team of developers. For example, if your team was working with several types of Arduino-compatible development boards, you can create an image containing the Arduino command line toolkit and a dumb-terminal emulation program for serial communication as your baseline development image. You can then use that base Arduino development image to create new variant images for each of the development boards that require specific custom drivers.
Docker's layered file system makes efficient use of space in this situation, since only the unique layers are stored when a new image is created. The docker history command displays a list of the various layers making up an image, including the instruction that created each layer and the size of the layer, such as:
IMAGE CREATED CREATED BY SIZE 3ca43a9a4095 20 hours ago apt-get install -y git 37.64 MB
The layers are cached, so you can quickly try out new variations of the image. For example, you might want to apply updates to drivers or development tools. Layers also make it painless to roll back to an earlier version of the image and try a different approach if something goes wrong.
If the changes to your development environment are successful, the new image can be pushed to your private team registry, to rapidly propagate the changes to the rest of the team. When each team member pulls the updated image, they'll already have most of the layers cached, so getting the new image up and running will typically take only a few seconds to a few minutes for Docker to run through the instructions to create the new layers. Compared to pushing and pulling snapshots of VM images or building entire images from scratch for each variation, layers save storage space but also more importantly your developer's time.
Deploying containers to IoT devices
The appeal of a Dockerized application is that after you have built an image, you can ship and run it almost anywhere. If your IoT device runs Linux, you might be able to deploy containers directly on your device. Despite the limited system resources that are typically available on such IoT devices, it is feasible to deploy Docker containers because their runtime overhead is almost zero. You can even run multiple containers on your device, for example, if you wanted to run different versions of your application side-by-side for comparison.
Docker requires a modern Linux kernel with support for kernel namespaces and control groups, so suitable base images might not yet be available for your preferred IoT device. You can expect to see more Linux-capable IoT devices being developed as the price point of suitable SoCs continues to drop (for example, the $9 C.H.I.P. development board); then, a wider choice of Linux distributions targeting these platforms will become available.
Docker Hub contains a number of ARM-based images for popular hobbyist SBCs like the Raspberry Pi and BeagleBone Black. If you run containers directly on your device, you can avoid having to constantly flash your device or overwrite the entire firmware. Instead you can build new images, pull them and run them within the host OS on the device, and rapidly prototype and experiment with bleeding-edge libraries and firmware to get the most out of your IoT device.
IoT developers face the challenge of keeping a network of connected devices updated and running on potentially unreliable and low-bandwidth wireless connections. Also, they must consider how to maintain the security of sensitive and highly personal data that many of these devices collect. IoT devices might only be connected sporadically, with some devices set to sleep on different schedules to conserve power. Other IoT devices might be connected by using a mesh network topology where only parts of the network are reachable at any given time. Push-based updates are likely to fail when network connectivity drops out, and the danger of applying incomplete or potentially corrupted updates is that devices can enter an inconsistent state or be rendered inoperable.
Docker provides a possible solution to this update problem. A device that issues a pull request for the latest version of its application image will be sent just the image diffs over the air rather than the entire image. Diff-based updates will complete much more quickly, which reduces the amount of time that the device needs to be connected and reduces the probability of failure, thus putting less stress on low-bandwidth networks. This makes it possible for updates to be applied more frequently.
Some IoT devices allow limited user interaction through physical buttons or small touchscreens, however, for many current generation IoT devices the primary means of user interaction is with mobile applications. It should come as no surprise that Docker containers are increasingly being adopted to speed up automated testing, continuous integration, and delivery of mobile applications.
Integrating IoT devices with the cloud
Deploying applications to smart devices and mobile devices is only part of the Internet of Things development story. Many of the current generation of IoT devices publish sensor data and events to cloud services.
Containers are designed for running cloud services. However, containers are ephemeral, so any data written to the file system within a running database container should be considered temporary; that is, the data will be lost if the container is shut down. In a sense, using containers forces you to establish the good habit of developing stateless cloud applications. Any data to be persisted should be stored using a data volume. Data volumes persist after a container shuts down, and can be shared between multiple containers.
As the number of connected devices increases, cloud-based applications for the IoT will need to scale to handle the volume of data being generated. Fortunately, we can take advantage of the growing ecosystem of orchestration tools built around Docker for developing scalable cloud applications, including the recently announced Docker Machine, Swarm, and Compose tools. These tools are only recommended for development environments at present, but Docker is working on making them production-ready for a number of cloud platforms, including EC2 Container Services, Microsoft Azure, and IBM Bluemix. IBM Bluemix also currently supports running groups of containers for load balancing and failover.
To meet the predicted demand for applications for the Internet of Things, developers will need to adopt tools and practices to enable them to rapidly develop IoT applications and services to run on smart devices, mobile devices, and in the cloud. With the ability to ship and run applications anywhere, with minimal runtime overhead, and with a layered file system that results in lightweight portable images and fast image builds, Docker containers are a great tool for IoT developers.
- Learn more about Docker and the difference between hypervisor-based virtualization and container-based virtualization
- Learn more about IBM Containers on the IBM Bluemix platform
- Why not try developing your next IoT project with containers?
- Get started with the Docker documentation.
- Learn how to
install the Docker tools so that you can run containers on your local
machine. Check out the Docker
documentation for details. If you are running OS X or Windows,
you'll also need to install
boot2docker, a lightweight Linux VM that runs your Docker containers.
- Try out the Beta release of Swarm, a native clustering tool for Docker
- Exploring IBM Bluemix (David Barnes, developerWorks, September 2013): Join IBM's David Barnes as he demonstrates IBM Bluemix, an in-the-cloud platform that provides the cloud application capabilities that will power the next generation of cloud applications and services.
- Internet of Things (IoT): Learn more about IoT on developerWorks.
- Build your own wearable with IBM Watson IoT Platform and IBM Bluemix (Mike Spisak and Rhonda Childress, developerWorks, February 2015); Build a hybrid mobile app that connects to a wearable device and sends sensor data from the device to the cloud.
- Explore MQTT and the Internet of Things service on IBM Bluemix (Chun Bin Tang, developerWorks, February 2015): See how MQTT works. Easily build applications using the IoT service and a sample Java app or the Node-RED editor.