Upgrading to version 1.0.7.8

Upgrade to version 1.0.7.8 is performed by IBM Support.

Before you begin

Before you start the upgrade, from /opt/ibm/appliance/platform/apos-comms/customer_network_config/ansible directory, you must run:

ANSIBLE_HASH_BEHAVIOUR=merge ansible-playbook -i ./System_Name.yml playbooks/house_config.yml --check -v

If any changes are listed in --check --v, ensure that they are expected. If they are unexpected, you must edit the YAML file so that it contains only the expected changes. You might rerun this command as necessary until you see no errors.

About this task

Upgrade to 1.0.7.8 is supported only for systems on 1.0.7.6 and above.

Only the system bundle is upgraded in 1.0.7.8. There is no need to download the following packages:

icpds_vm
icpds_rhos_repo
icpds_services
icpds_services_addon_cyclops

Draft comment: anna.bogusz@pl.ibm.com
do we need any prerequisites from 1076 updgrade or were these issues fixed? https://ibmdocs-test.mybluemix.net/docs/en/SS5FPD_1.0.0_test?topic=system-upgrading-version-1076

Procedure

Connect to node e1n1 via the management address and not the application address or floating address.
Verify that e1n1 is the hub:
1. Check for the hub node by verifying that the dhcpd service is running:
```
systemctl is-active dhcpd
```
2. If the dhcpd service is running on a node other than e1n1, bring the service down on that other node:
```
systemctl stop dhcpd
```
3. On e1n1, run:
```
systemctl start dhcpd
```
Download the system bundle from Fix Central and copy it to /localrepo on e1n1.

Note: The upgrade bundle requires a significant amount of free space. Make sure you delete all bundle files from previous releases.
From the /localrepo directory on e1n1, run:
```
mkdir 1.0.7.8_release
```
and move the system bundle into that directory.
The directory that is used here must be uniquely named - for example, no previous upgrades on the system can have been run out of a directory with the same name.

Optional: Run upgrade details to view details about the specific upgrade version:

apupgrade --upgrade-details --upgrade-directory /localrepo --use-version 1.0.7.8_release --bundle system

Before you start the upgrade process, depending on your requirements:
- Run the preliminary checks with --preliminary-check option:
```
apupgrade --preliminary-check --upgrade-directory /localrepo --use-version 1.0.7.8_release --bundle system
```
  if you just want to check for potential issues and cannot accept any system disruptions. This check is non-invasive and you can rerun it as necessary. You can expect the following output after you run the preliminary checks.
```
All preliminary checks complete
Finished running pre-checks.
```
- Optional: Run the preliminary checks with --preliminary-check-with-fixes option:
```
apupgrade --preliminary-check-with-fixes --upgrade-directory /localrepo --use-version 1.0.7.8_release --bundle system
```
  if you want to check for potential issues and attempt to automatically fix those. Run it if you can accept your system to be disrupted as this command might cause the nodes to reboot.
Draft comment: anna.bogusz@pl.ibm.com
is this required in 1078?
Optional: Upgrade the apupgrade command to get the new command options:
```
apupgrade --upgrade-apupgrade --upgrade-directory /localrepo --use-version 1.0.7.8_release --bundle system
```
The value for the --use-version parameter is the same as the name of the directory you created in step 4.
Open support ticket to have the system upgraded by IBM Support.
Draft comment: anna.bogusz@pl.ibm.com
THE STEPS THAT FOLLOW WILL BE HIDDEN FROM PUBLIC KC AND WILL ONLYBE SHOWN IN SUPPORT KC

Results

After the upgrade is complete, some of the following alerts might be opened in the system:

| 439         | SW_NEEDS_ATTENTION         | SW    | Openshift node is not ready                                   | YES      |
| 440         | SW_NEEDS_ATTENTION         | SW    | Openshift service is not ready                                | YES      |
| 446         | SW_NEEDS_ATTENTION         | SW    | ICP4D service is not ready                                    | YES      |
| 451         | SW_NEEDS_ATTENTION         | SW    | Webconsole service is not ready                               | YES      |
| 460         | SW_NEEDS_ATTENTION         | SW    | Portworx component is not healthy

Close them manually with ap issues --close <alert_id>

As part of the upgrade process, VMs are disabled on all nodes and they are shut down. They are expected to stay in the shut off state in 1.0.7.8.

[root@gt01-node1 ~]# for node in `/opt/ibm/appliance/platform/xcat/scripts/xcat/display_nodes.py`; do echo ${node}; ssh $node virsh list --all; done
e1n1
 Id    Name                           State
----------------------------------------------------
 -     e1n1-1-control                 shut off

e1n2
 Id    Name                           State
----------------------------------------------------
 -     e1n2-1-control                 shut off

e1n3
 Id    Name                           State
----------------------------------------------------
 -     e1n3-1-control                 shut off

e1n4
 Id    Name                           State
----------------------------------------------------
 -     e1n4-1-worker                  shut off

e2n1
e2n2
e2n3
e2n4

In 1.0.7.8, the Netezza web console runs on one of the three control nodes (or on a connector node if installed). There are two docker containers required for operation of the Netezza console: cyclops and the associated influxdb container. Container images are installed on all control nodes for high availability. When a control node goes out of service, Platform Manager starts the cyclops and influxdb containers on another control node (or a connector node).

[root@gt01-node1 ~]# for node in `/opt/ibm/appliance/platform/xcat/scripts/xcat/display_nodes.py --control`; do echo ${node}; ssh $node docker ps -a | grep -E 'cyclops|influxdb'; done
e1n1
0d5c48c43d1d        cyclops:4.0.2-20220131b16309-x86_64   "/scripts/start.sh"      12 hours ago        Created                              cyclops
0166a1a80a35        influxdb:latest                       "/entrypoint.sh in..."   12 hours ago        Up 6 hours                     0.0.0.0:8086->8086/tcp   influxdb
e1n2
ede85f799cd5        cyclops:4.0.2-20220131b16309-x86_64   "/scripts/start.sh"      12 hours ago        Exited (137) 12 hours ago                       cyclops
0ade5a9fedb8        influxdb:latest                       "/entrypoint.sh in..."   12 hours ago        Exited (0) 12 hours ago                         influxdb
e1n3
a9e6d1d07a95        cyclops:4.0.2-20220131b16309-x86_64   "/scripts/start.sh"      12 hours ago        Exited (137) 12 hours ago                       cyclops
d69a463388e5        influxdb:latest                       "/entrypoint.sh in..."   12 hours ago        Exited (0) 12 hours ago                         influxdb
[root@gt01-node1 ~]#

The ap version -s container reflects the web console version:

[root@gt26-node1 ~]# ap version -s
Appliance software version is 1.0.7.8

All component versions are synchronized.

+-----------------------------+--------------------------------------------------------------------+
| Component Name              | Version                                                            |
+-----------------------------+--------------------------------------------------------------------+
| Appliance platform software | 1.0.7.8-20220118033947b15184                                       |
.
.
.
| cyclops                     | cyclops:4.0.2-20220131b16309