Upgrading Docker

Upgrade Docker from version 18.06.2 to version 19.03.11 in your IBM® Cloud Private cluster that was installed by using the 3.2.0.2003 fix pack.

First, upgrade Docker from version 18.06.2 to 18.09.7. Then, upgrade Docker to version 19.03.11.

Upgrading Docker when the boot node is not a master node

If your boot node is not a master node, complete these steps to upgrade Docker:

Upgrade Docker to version 18.09.7

Complete these steps to upgrade Docker to version 18.09.7:

  1. Log in to your boot node as a cluster administrator.

  2. Create a k8s.yaml file in the <installation_directory>/cluster/ directory. The file must have the following content:

    # Licensed Materials - Property of IBM
# 5737-E67
# @ Copyright IBM Corporation 2016, 2018 All Rights Reserved
# US Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

---

- name: Waiting for Kubernetes to start
 shell: |
   status=$(kubectl version &>/dev/null && echo yes || echo no)
   if [[ "$status" == "no" ]]; then
     echo "Kubernetes is not ready, check the following logs"
     echo "---------------------- Kubernetes apiserver logs ----------------------"
     for id in $(docker ps -aq --filter name=k8s_apiserver)
     do
       docker logs $id --tail=30
     done
     echo
     echo "---------------------- kubelet logs ----------------------"
     journalctl -u kubelet.service --no-pager --lines=30
   else
     echo $status
   fi
 args:
   executable: /bin/bash
 delegate_to: "{{ master_nodes[0] }}"
 register: result
 until: result.stdout == "yes"
 retries: 10
 delay: 60

  3. Edit the <installation_directory>/cluster/config.yaml file and change the docker_version.

    ## Install/upgrade docker version
docker_version: 18.09.7

  4. Download the Docker binary file from the IBM Passport Advantage® Opens in a new tab website.

    • For Linux® x86_64, download icp-docker-18.09.7_x86_64.bin.
    • For Linux® on Power® (ppc64le), download icp-docker-18.09.7_ppc64le.bin.
    • For Linux® on IBM® Z and LinuxONE, download icp-docker-18.09.7_s390x.bin.

  5. Create the runtime-engine directory under the cluster directory.

    mkdir <installation_directory>/cluster/runtime-engine/

  6. Copy the Docker binary file to the runtime-engine directory.

    cp <Docker-binary-file-path>/<Docker-binary-file-name> <installation_directory>/cluster/runtime-engine/

  7. Change to the cluster directory.

    cd <installation_directory>/cluster/

  8. Upgrade Docker by running the following command:

    sudo docker run --net=host -t -e LICENSE=accept -v "$(pwd)":/installer/cluster -v "$(pwd)/k8s.yaml":/installer/playbook/roles/waitfor/tasks/k8s.yaml ibmcom/icp-inception-amd64:3.2.0-ee upgrade-docker

Upgrade Docker to version 19.03.11

Complete these steps to upgrade Docker to version 19.03.11:

  1. Log in to your boot node as a cluster administrator.

  2. Edit the <installation_directory>/cluster/config.yaml file and change the docker_version.

    ## Install/upgrade docker version
docker_version: 19.03.11

  3. Download the Docker binary file from the IBM Passport Advantage® Opens in a new tab website.

    • For Linux® x86_64, download icp-docker-19.03.11_x86_64.bin.
    • For Linux® on Power® (ppc64le), download icp-docker-19.03.11_ppc64le.bin.
    • For Linux® on IBM® Z and LinuxONE, download icp-docker-19.03.11_s390x.bin.

  4. Copy the Docker binary file to the runtime-engine directory.

    cp <Docker-binary-file-path>/<Docker-binary-file-name> <installation_directory>/cluster/runtime-engine/

  5. Upgrade Docker by running the following command:

    sudo docker run --net=host -t -e LICENSE=accept -v "$(pwd)":/installer/cluster ibmcom/icp-inception-amd64:3.2.0-ee upgrade-docker

Upgrading Docker when the boot node is a master node

If your boot node is also one of the master nodes, complete these steps to upgrade the Docker version:

Upgrade Docker to version 18.09.7

Complete these steps to upgrade Docker to version 18.09.7:

  1. Cordon the master node.

    kubectl cordon <master-node-IP-address>

  2. Drain the node. 

    kubectl drain <master-node-IP-address> --ignore-daemonsets --delete-local-data --force --timeout=600s

  3. Upgrade Docker on the master node. Use the Docker version 18.09.7 binary file. For example, icp-docker-18.09.7_x86_64.bin.

    ./<installation_directory>/cluster/runtime-engine/<Docker-binary-file-name> --upgrade

    Following is a sample output:

    [20-08-13 19:23:38] ACTION: upgrade
[20-08-13 19:23:38] DOCKER_ENV:
[20-08-13 19:23:38] DOCKER_CONFIG:
[20-08-13 19:23:38] Upgrading docker
[20-08-13 19:23:39] Extracting docker.tar.gz
[20-08-13 19:23:45] Upgrading docker package on Ubuntu 18.04
./runtime-engine/icp-docker-18.09.7_x86_64.bin: line 271: rpm: command not found
[20-08-13 19:25:07] Restarting docker service
[20-08-13 19:25:10] Upgrade docker successfully.

  4. Uncordon the master node.

    kubectl uncordon <master-node-IP-address>

    Following is a sample output:

    node/10.11.18.99 uncordoned

After you upgrade, verify the Docker version on the nodes. If some pods are not successfully created, you must manually delete these pods to re-create them.

kubectl get nodes -o wide

Following is a sample output:

NAME           STATUS   ROLES                          AGE   VERSION               INTERNAL-IP    EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION       CONTAINER-RUNTIME
10.11.18.100   Ready    worker                         16h   v1.13.12+icp-ee2003   10.11.18.100   <none>        Ubuntu 18.04.5 LTS   4.15.0-112-generic   docker://18.9.7
10.11.18.103   Ready    worker                         16h   v1.13.12+icp-ee2003   10.11.18.103   <none>        Ubuntu 18.04.5 LTS   4.15.0-112-generic   docker://18.9.7
10.11.18.99    Ready    etcd,management,master,proxy   17h   v1.13.12+icp-ee2003   9.30.183.77    <none>        Ubuntu 18.04.5 LTS   4.15.0-112-generic   docker://18.9.7

Upgrade Docker to version 19.03.11

Complete these steps to upgrade Docker to version 19.03.11:

  1. Cordon the master node.

    kubectl cordon <master-node-IP-address>

  2. Drain the node. 

    kubectl drain <master-node-IP-address> --ignore-daemonsets --delete-local-data --force --timeout=600s

  3. Upgrade Docker on the master node. Use the Docker version 19.03.11 binary file. For example, icp-docker-19.03.11_x86_64.bin.

    ./<installation_directory>/cluster/runtime-engine/<Docker-binary-file-name> --upgrade

    Following is a sample output:

    [20-08-13 19:23:38] ACTION: upgrade
[20-08-13 19:23:38] DOCKER_ENV:
[20-08-13 19:23:38] DOCKER_CONFIG:
[20-08-13 19:23:38] Upgrading docker
[20-08-13 19:23:39] Extracting docker.tar.gz
[20-08-13 19:23:45] Upgrading docker package on Ubuntu 18.04
./runtime-engine/icp-docker-19.03.11_x86_64.bin: line 271: rpm: command not found
[20-08-13 19:25:07] Restarting docker service
[20-08-13 19:25:10] Upgrade docker successfully.

  4. Uncordon the master node.

    kubectl uncordon <master-node-IP-address>

    Following is a sample output:

    node/10.11.18.99 uncordoned

After you upgrade, verify the Docker version on the nodes. If some pods are not successfully created, you must manually delete these pods to re-create them.

kubectl get nodes -o wide

Following is a sample output:

NAME           STATUS   ROLES                          AGE   VERSION               INTERNAL-IP    EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION       CONTAINER-RUNTIME
10.11.18.100   Ready    worker                         16h   v1.13.12+icp-ee2003   10.11.18.100   <none>        Ubuntu 18.04.5 LTS   4.15.0-112-generic   docker://19.3.11
10.11.18.103   Ready    worker                         16h   v1.13.12+icp-ee2003   10.11.18.103   <none>        Ubuntu 18.04.5 LTS   4.15.0-112-generic   docker://19.3.11
10.11.18.99    Ready    etcd,management,master,proxy   17h   v1.13.12+icp-ee2003   9.30.183.77    <none>        Ubuntu 18.04.5 LTS   4.15.0-112-generic   docker://19.3.11