Defining Docker Observer jobs

Using the Docker Observer functionality, you can discover Docker network resources, including Docker Swarm clusters, and then visualize this data as a topology view in the Agile Service Manager UI. In addition, it also discovers Docker clusters managed by Docker UCP.

Before you begin

Important: The Docker Observer supports Docker version 3.1.0.

Note: Docker UCP v3.1.0 supports only TLS 1.2 for SSL negotiation and has removed support for TLS 1 and TLS 1.1.

The Docker Observer is installed as part of the core installation procedure.

Remember: Swagger documentation for the observer is available at the following default location: https://<your host>/1.0/docker-observer/swagger

About this task

The Docker Observer performs a single load job, which performs a one-off discovery of the Docker network.

The job definition indicates whether to connect to a local Docker on the same (UNIX) host as the observer using the unix_socket parameter, or to a remote Docker using the host and port parameters.

Local docker

The default, if the job parameters are empty, is to try to connect to a UNIX socket at /var/run/docker.sock

If the location of the UNIX socket differs, the full path can be given in the unix_socket parameter. The host and port parameters must not be supplied.

In either case, the socket must be accessible.

When the observer is running within the docker container to be monitored, /var/run/docker.sock must be available within the container. For example:

volumes: 
/var/run/docker.sock:/var/run/docker.sock

Remote docker

The host and port parameters of the job can be used to identify the TCP port that Docker can be reached on. The unix_socket parameter must not be supplied.

Docker is not accessible via TCP by default. To enable it, edit the docker.service file. On RedHat, this is available in /usr/lib/systemd/system. Amend the ExecStart option under the Service section to include a -H option. For example, to make it available externally on port 2375, you could add -H tcp://0.0.0.0:2375.

Note: If you want to continue to be able to access Docker via the default socket, for example if the Docker Observer container needs access, or if you want to be able to perform docker ps -a rather than docker -H tcp://0.0.0.0:2375 ps -a, then you need to also list it in the same line, as on the following example:

-H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock

You must reload the configuration:

sudo systemctl daemon-reload
sudo systemctl restart docker

Tip: If this fails to start Docker, and a Unix socket (or no socket at all) was specified, check that no directory with that name exists. If you start up docker with just a TCP socket and no Unix socket, this creates a /var/run/docker.sock directory, which you must delete after Docker is stopped, so that you can restart with access via that Unix socket.

docker_observer_common.sh

The configuration file you use to customize Docker Observer settings.

The parameters defined here are then used by the docker_observer_load_start.sh script to trigger the Docker Observer job.

You can use the view_all and exclude_containers parameters to filter the scope of observations. These parameters are arrays or lists that can accept multiple values.

view_all: Use this parameter to force modeling of all containers, tasks and images.; By default, only running containers, running tasks, and images currently in use by modeled containers are modeled.
exclude_containers: Use this parameter to filter out containers that are not of interest, based on regular expression matches against the container name.

Swagger UI usage examples

Using the Docker Observer, you can discover the following Docker resources:

Remote Docker network resources via HTTP through TCP port exposure.
Example:

{
  "unique_id": "my job",
  "type": "load",
  "parameters": {
    "host": "1.2.3.4",
    "port": 2375
  }
}

Remote Docker network resources with HTTPS using a certificate.
Example:

{
  "unique_id": "my job",
  "type": "load",
  "parameters": {
    "host": "1.2.3.4",
    "port": 2375,
    "username": "username",
    "password": "password",
    "docker_ssl_certificate": "certificate_file_name.crt",
    "docker_ssl_truststore_file": "truststore_file_name.jks",
    "password_ssl_truststore_file": "truststore_password"
  }
}

Remote Docker network resources with HTTPS using certificate and truststore.
Example:

{
  "unique_id": "my job",
  "type": "load",
  "parameters": {
    "host": "1.2.3.4",
    "port": 2375,
    "username": "username",
    "password": "password",
    "docker_ssl_certificate": "certificate_file_name.crt",
    "docker_ssl_truststore_file": "truststore_file_name.jks",
    "password_ssl_truststore_file": "truststore_password"
  }
}

Remote Docker network resources with HTTPS using truststore.
Example:

{
  "unique_id": "my job",
  "type": "load",
  "parameters": {
    "host": "1.2.3.4",
    "port": 2375,
    "username": "username",
    "password": "password",
    "docker_ssl_truststore_file": "truststore_file_name.jks",
    "password_ssl_truststore_file": "truststore_password"
  }
}

Procedure

Edit the docker_observer_common.sh config file as required.
The Docker Observer supports multiple types of Docker configurations. Edit or populate the following parameters for different docker configurations:

Local Docker

Empty parameters

Remote Docker with HTTP

Populate host and port

Remote Docker with HTTPS via certificate

The certificate will be added to the named truststore

Populate host, port, username, password, docker_ssl_certificate, docker_ssl_truststore_file and password_ssl_truststore_file

Remote Docker with HTTPS via truststore

The truststore must contain the certificate

Populate host, port, username, password, docker_ssl_truststore_file and password_ssl_truststore_file
Encryption requirement: All jobs require passwords in encrypted form. To encrypt password and password_ssl_truststore_file, run the encrypt_password.sh script in the $ASM_HOME/bin directory:
```
./bin/encrypt_password.sh 
```
Enter and then confirm the password. The encryption utility will return an encrypted password.
SSL requirements: To acquire SSL certificates and build SSL truststores, use the relevant instructions in the following section: Configuring observer job security

To start a Docker Observer Load job, use the following command:

$ASM_HOME/bin/docker_observer_load_start.sh

Usage examples for starting jobs:

Default job

$ASM_HOME/bin/docker_observer_load_start.sh

Local Docker

env unique_id=My job name $ASM_HOME/bin/docker_observer_load_start.sh

Remote Docker with HTTP

env unique_id=My job name host=1.2.3.4 port=2375 $ASM_HOME/bin/docker_observer_load_start.sh

Remote Docker with HTTPS via certificate

env unique_id=My job name host=1.2.3.4 port=2375 username=username password=password docker_ssl_certificate=certificate_file_name.crt docker_ssl_truststore_file=truststore_file_name.jks password_ssl_truststore_file=truststore_password $ASM_HOME/bin/docker_observer_load_start.sh

Remote Docker with HTTPS via truststore

env unique_id=My job name host=1.2.3.4 port=2375 username=username password=password docker_ssl_truststore_file=truststore_file_name.jks password_ssl_truststore_file=truststore_password $ASM_HOME/bin/docker_observer_load_start.sh

Results

The script triggers the Docker Observer Load job, which performs a one-off discovery of the Docker network you have specified.

What to do next

You can also use the following scripts:

docker_observer_load_stop.sh: Stops the job
docker_observer_job_list.sh: Lists the current job status
docker_observer_log_level.sh: Sets the log level

Tip: For regular status updates, run the default Docker Observer job via a cron job.

Remember: As an alternative to being configured using the Observer Configuration UI, observer jobs have scripts to start and stop all available jobs, to list the status of a current job, and to set its logging levels. These scripts can be run with -h or --help to display help information, and with -v or --verbose to print out the details of the actions performed by the script, including the full cURL command. For the on-prem version of Agile Service Manager, observer scripts are configured for specific jobs by editing the script configuration files.