Docker overview and scope

Docker is used to build, ship, and run distributed applications. You can configure your service instances to run in Docker containers.

A Docker container allows an application to be packed together with its dependencies into a portable virtual package that can run with multi-platform support, isolation, and resource limits applied.

The main benefits of running a service instance in a Docker container are as follows:

Deployment: Application can be packed together to a portable Docker image, that can run on different platforms.
Isolation: Containers isolate applications from each other and the underlying infrastructure while providing an added layer of protection for the application.

The following diagram shows how IBM® Spectrum Symphony service instances run directly on compute hosts without Docker:

Platform Symphony service instances run directly on compute hosts with Docker

The following diagram shows how IBM Spectrum Symphony service instances run inside containers when Docker is enabled. The SIM starts the Docker containers instead of directly starting the SI processes.

Platform Symphony service instances run inside containers when Docker is enabled. The SIM starts the Docker containers instead of directly starting the SI processes.

IBM Spectrum Symphony supports Docker for both SOA and MapReduce applications. One IBM Spectrum Symphony application can accept workload from multiple tenants where each tenant has its own OS user (the tenant OS user is not the same as the job or session OS submission user). Each tenant's OS user has access to specific file mounts and should not be able to see other tenant’s mounts. Ideally, the Docker Container should be run as the tenant's OS user to avoid modifying other tenants' data.

Service instance start-up

Here is the sequence IBM Spectrum Symphony follows to start service instances in Docker containers:

SIM starts the Docker Controller. The service instance starts in a container and connects to SIM.
Docker Controller sends a request to the Docker daemon to start a container and mount host data volumes to the container.
Service instance starts in the container and connects to SIM.

Docker security

The SIM runs as consumer execution user, so this user should be added to the user group that is allowed to use Docker. Containers are started as unprivileged by the SIM to make the container more secure.

The user inside the container is configurable. You can configure a specific user, which should be a built-in user of the Docker image, to start your command inside the container. By default, IBM Spectrum Symphony uses the consumer OS execution user as the container user. New files created by a service instance are owned by the consumer OS execution user.

Docker versions

The application profile contains a configuration parameter called dockerVersions, which allows you to specify the Docker version numbers that the applications can run with. The SSM merges the requirement for the specified Docker versions into the resource requirement for compute resources, which means the SSM will only request resources where the Docker daemon with the requested Docker version number is running.

Container environment

A service instance running in a Docker container receives the same environment variables as it would if it were running on the local host directly.

Container working directory

The container working directory is set to the service workDir so that the service instance log file can write to the host OS.

Log directory

Logs for the Docker Controller are written to the location specified by logDirectory in the Service section of the application profile. If logDirectory is not configured, the default location is $SOAM_HOME/work.

Limiting resources

When this feature is enabled, memory and CPU limits related to the cgroup file are managed by Docker.

Docker has two options to constrain memory usage of a container:

memory (mapped to physicalMemorylimit in the application profile)
memory-swap (mapped to virtualMemorylimit in the application profile)

To limit CPU resources, IBM Spectrum Symphony uses cpuLimit, which is configured in the application profile. For Docker containers, IBM Spectrum Symphony maps cpuLimit to Docker's cpu-shares.