Platform Manager certificate patch release notes

As platform manager certificates expire after 8 January 2022, you must update them before that date. Without applying this patch in time, you will not be able to run ap commands. After the patch is successfully applied, the certificates expiration date is extended to 100 years.

There are two types of certificates, both of which must be upgraded:
  • REST certificates that are used for externally accessible REST API (for example: ap commands use this API).
  • Cluster certificates that are used for internal platform management communication, no endpoint accessible externally uses them.

Normally, the certificates are updated automatically during upgrades. However, if the system was not upgraded for a long time, the certificates expiration date might be missed and cause the platform manager to stop working. You must run regenerate_certificates.py script to renew the Platform Manager certificates.

The patch also includes an update for the Call Home TrustList.jks file. The previously issued trust file expires in June 2022.

When you run the script, you can select not to update the certificates and only update the Call Home file, but it is not recommended.

Important: Before 8 January 2022, you must either apply the certificate patch, or upgrade to version 1.0.26.1, which also includes the certificate patch and the trust file. If you plan to upgrade to 1.0.26.1 before 8 January 2022, there is no need to apply this patch.

Before you begin

  • The patch is applicable to any 1.0.x Integrated Analytics System version.
  • The estimated run time is 5-15 minutes, depending on the system size. Platform Manager is stopped, but the applications remain online. Database remains online.
  • The patch is executed by running an interactive regenerate_certificates.py script that guides you through the process.
  • If the certificate patch is applied and you want to upgrade to any later 1.0.x version, the platform management certificates (both, REST and cluster) are retained. However, Call Home TrustList.jks file gets overwritten in the upgrade process. You will have to rerun the script to update only the Call Home TrustList.jks file after the upgrade.
  • If applying the certificate patch after the certificates expired, see Applying the patch on a system with expired certificates.
  • If your system has any nodes disabled due to not being reachable, resolve this issue and make the nodes reachable again before applying the patch. Otherwise, contact Support team to get the patch applied on a system with unreachable nodes.
Run the commands as root. Either log in as root directly, or use the command su -. The su root command does not work and causes the process to fail.

Procedure

  1. Download the 1.0.x.x.platform_management_certificate_patch.IF1-IM-IIAS-fpxxx package from Fix Central.
  2. Once the tar file is downloaded, untar the file using tar -xvf command.
    Example:
    [root@node0101 localrepo]# tar -xvf certificates_regeneration-release-1.0.1.0-noarch.tar
certificates_regeneration/
certificates_regeneration/regenerate_certificates.py
certificates_regeneration/TrustList.jks
    After the file is untared, the directory certificates_regeneration is created, containing both regenerate_certificates.py and TrustList.jks.
  3. Run: 
    cd certificates_regeneration
  4. Run python regenerate_certificates.py without any parameters.
  5. Wait for the nodes check to complete.
  6. When prompted to update the REST certificate:
    • Type y to update the REST certificate.
    • Type n not to update and press enter. Go to 8.
  7. When prompted, decide whether you want to apply your custom REST certificate (if you have it) or the default one:
    • Type y to provide the path to the custom REST certificate and key file. The custom REST certificate is applied.
    • Type n if you want to use the default REST certificate instead. The script then runs a default certificate expiry check and prompts you to type y to confirm the update. The default REST certificate is applied.
  8. When prompted to update the cluster certificates:
    • Type y to confirm. The cluster certificates are applied.
    • Type n not to update and press enter.
  9. The script automatically updates the TrustList.jks file. Wait for the script to start the platform manager and exit.
    Example:
    [root@node0101 ~]# python regenerate_certificates.py
Started certificate regeneration script
Checking nodes list...
node0101-fab, node0102-fab, node0103-fab
Checking nodes reachability...
Checking reachability of node0101-fab... ok
Checking reachability of node0102-fab... ok
Checking reachability of node0103-fab... ok
All the nodes are reachable, proceeding
Checking system state... Stopped
Checking for custom REST certificate... not found
Checking default certificate expiry date...
Default certificate expires on 2022-01-08 00:00:03 (in 80 days).
Do you want to update it? y/[n]: y
Certificate can be either regenerated automatically with default data or you can provide new custom one.
Do you want to provide custom certificate? y/[n]: n
Regenerating certificate automatically
Generating new REST certificate in temp location... done
Sending certificate to temporary locations on all the nodes
Sending to node0101-fab... done
Sending to node0102-fab... done
Sending to node0103-fab... done
Validating REST certificate on node0101-fab... ok
Validating REST certificate on node0102-fab... ok
Validating REST certificate on node0103-fab... ok
Validation was successful on all the nodes
Certificate for REST copied to nodes. Stopping platform manager services...
Platform management stopped, applying new certificates
Applying new certificates on node0101-fab... done
Applying new certificates on node0102-fab... done
Applying new certificates on node0103-fab... done
REST certificate update was successful
Checking for custom cluster certificates... not found
Checking default certificates expiry date...
Default certificates expire on 2022-11-19 17:06:12 (in 13 months).
Cluster certificates can only be regenerated automatically, as they are internal ones, not exposed.
Do you want to regenerate them? y/[n]: y
Regenerating certificates
Generating new cluster certificates in temp location... done
Sending certificate to temporary locations on all the nodes
Sending to node0101-fab... done
Sending to node0102-fab... done
Sending to node0103-fab... done
Validating cluster certificates on node0101-fab... ok
Validating cluster certificates on node0102-fab... ok
Validating cluster certificates on node0103-fab... ok
Validation was successful on all the nodes
Certificate for cluster copied to nodes. Stopping platform manager services...
Platform management stopped, applying new certificates
Applying new certificates on node0101-fab... done
Applying new certificates on node0102-fab... done
Applying new certificates on node0103-fab... done
Cluster certificates update was successful
Checking whether callhome container is running... yes
Updating callhome trust list file...
Backing up /opt/ibm/appliance/storage/platform/ras/callhome/ecc-root/config/TrustList.jks to /opt/ibm/appliance/storage/platform/ras/callhome/ecc-root/config/TrustList.jks.bak
The TrustList.jks has been copied to /opt/ibm/appliance/storage/platform/ras/callhome/ecc-root/config/TrustList.jks
Updated, running apstart -p...
Successfully activated platform
Script is done, exiting
[root@node0101 ~]#
  10. Verify that certificates were properly updated by running the following commands and checking the Not After field:
    • REST certificates:
      [root@node0101 ~]# openssl s_client -connect node0101-fab:5001 2>/dev/null | openssl x509 -text -noout
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 1 (0x1)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C=US, CN=PlatformManager, ST=New York, L=Armonk, O=International Business Machines Corporation, OU=Analytics
        Validity
            Not Before: Jan  1 00:00:00 1970 GMT
            Not After : Sep 25 15:33:13 2121 GMT
        Subject: C=US, CN=PlatformManager, ST=New York, L=Armonk, O=International Business Machines Corporation, OU=Analytics
        Subject Public Key Info:
            ...
    • Cluster certificates:
      [root@node0101 ~]# openssl s_client -connect node0101-fab:5003 2>/dev/null | openssl x509 -text -noout
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 2 (0x2)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C=US, CN=PlatformManager, ST=New York, L=Armonk, O=International Business Machines Corporation, OU=Analytics
        Validity
            Not Before: Jan  1 00:00:00 1970 GMT
            Not After : Sep 25 15:34:32 2121 GMT
        Subject: C=US, CN=PlatformManager, ST=New York, L=Armonk, O=International Business Machines Corporation, OU=Analytics
        Subject Public Key Info:
            ...

Applying the patch on a system with expired certificates

If the certificates on your system already expired, you can still apply the patch. However, some workaround steps are required because Platform Manager is not fully operational in this state. After applying the procedure as described above, the system fails to come online and Db2 is down. To overcome the situation, you need to run the following steps:

Procedure

  1. Run: 
    apstop -v
  2. Run: 
    apstart -p
    to start the platform only.
  3. Run: 
    ap node
    to verify which nodes are disabled.
  4. Enable all of the disabled nodes by using: 
    ap node enable
  5. Run: 
    apstart -a
    to start the application.

Known issues

regenerate_certificates.py script fails with the following error: UnicodeEncodeError: 'ascii' codec can't encode character u'\u2018' in position 60: ordinal not in range(128)
Workaround:
  1. Add the following lines:
    reload(sys)
sys.setdefaultencoding('UTF8')
    in regenerate_certificates.py script after the import sys line.
  2. Rerun the script.