Upgrading Global Mailbox to v6.2.1.0

Upgrading to Global Mailbox v6.2.1.0 involves transitioning from JCEKS to PKCS12 due to JDK17 incompatibilities, requiring a hotfix to convert keystores and encrypted payloads. The process includes sequential node upgrades and utility runs to maintain functionality and data accessibility.

IBM JDK8 supports JCEKS as the keystore type and IBMJCE (IBMJCEPlus) as the keystore provider. Global Mailbox uses JCEKS keystore and IBMJCE(IBMJCEPlus) provider across all versions of the product. However, in GlobalMailbox 6.2.1.0 release, JDK17 does not support JCEKS keystore and IBMJCE (IBMJCEPlus) provider. Refer to the following for information on JDK17:

java version "17.0.14" 2025-01-21
IBM Semeru Runtime Certified Edition 17.0.14.0 (build 17.0.14+7) 
Eclipse OpenJ9 VM 17.0.14.0 (build openj9-0.49.0, JRE 17 Linux amd64-64-Bit 
Compressed References 20250121_885 (JIT enabled, AOT enabled)
OpenJ9 - 3c3d179854
OMR - e49875871  
JCL - 675a4171160 based on jdk-17.0.14+7)

Prerequisites

It is not recommended to perform a direct upgrade to GlobalMailbox v6.2.1.0 from your current version. Instead, follow these steps for a successful upgrade to 6.2.1.0:

Request a hotfix for your current version. For example, if you are using Global Mailbox version 6.1.2.7, request a hotfix for that version.
The hotfix approach is used to convert existing keystores of type JCEKS to the PKCS12 type, supported by JDK17.
The hotfix introduces two utilities that you must run for successful keystore conversions:
- PreUpgradeUtility: Converts all required keystores in Global Mailbox.
- regenerateEncryptedPayloads: Converts all inline encrypted payloads of type JCEKS to PKCS12.
Note: Running the above two utilities is mandatory during the rolling upgrade to the hotfix. Even if you do not have inline encrypted payloads in the system, you must not skip the regenerateEncryptedPayloads utility. It adds markers in the Cassandra database necessary for validations during the upgrade to Global Mailbox 6.2.1.0.

Upgrading to hotfix

In a two data center (DC) Global Mailbox cluster environment, with DC1 and DC2 each having two Sterling B2B Integrator + Global Mailbox nodes, follow these steps:

Trigger the Upgrade on DC1 Node1:
- Upgrade DC1Node1 to the hotfix. During this upgrade, file uploads, routing, and replication should continue to function correctly from the other three nodes.
- After the upgrade, file uploads from DC1Node1 will start failing, and issues will arise with the user lookup service and FTP/SFTP mailbox listing containing inline payloads on DC1Node1.
Obtain the application name and ID by running the following command:
```
select app_id, app_name from gatekeeper.applications;
```
Use this output for the --appName and --appID parameters in the subsequent steps.

Run one of the following commands, depending on your MQ SSL configuration, for the PreUpgradeUtility on DC1Node1:

MQ with SSL:

./appConfigUtility.sh PreUpgradeUtility --appName=B2Bi --appID=<app_ID> --dcName=<DC1_name> --qmgr=<QMGR> --channel=<ssl_channel> --connList="<IP(PORT)>" --appQueue=<queue_name> --mqUser=<mq_username> --mqPassword=<mq_password> --adminUser=<admin_username> --adminPassword=<admin_password> --enableSSL=true --trustStoreFile=<path_to_truststore.jks> --trustStorePassword=<truststore_password> --trustStoreType=JKS --cipherSpec=TLS_RSA_WITH_AES_128_CBC_SHA256 --keyStoreFile=<path_to_key.jks> --keyStoreType=JKS --keyStorePassword=<keystore_password> --authUser=gmbx_user --authPassword=password --encryption=true --test

Note: The value for --cipherSpec will depend on the cipher configuration set in MQ. For more information, see Enabling CipherSpecs.

MQ without SSL:

./appConfigUtility.sh PreUpgradeUtility --appName=B2Bi --appID=<app_ID> --dcName=<DC1_name> --qmgr=<QMGR> --channel=<ssl_channel> --connList="<IP(PORT)>" --appQueue=<queue_name> --mqUser=<mq_username> --mqPassword=<mq_password> --adminUser=<admin_username> --adminPassword=<admin_password> --enableSSL=false --authUser=gmbx_user --authPassword=password --encryption=true --test

Note: You must modify the parameter values to suit your environment.

After running this utility, the user lookup and file upload issues on DC1Node1 will be resolved. However, these issues will then appear on DC1Node2 and both nodes of DC2.

Upgrade DC1Node2 to Hotfix:
After upgrading, file uploads from both nodes of DC1 will function correctly, but file upload failures will continue on both nodes of DC2.

Upgrade and Configure DC2Node1:

Upgrade DC2Node1 to the hotfix.

If file uploads still fail from DC2Node1, run the following utility:

MQ with SSL:

./appConfigUtility.sh setMQConfig --appName=B2Bi --dcName=<DC2_name> --qmgr=<QMGR> --channel=<ssl_channel> --connList="<IP(PORT)>" --appQueue=<queue_name> --mqUser=<mq_username> --mqPassword=<mq_password> --adminUser=<admin_username> --adminPassword=<admin_password> --enableSSL=true --trustStoreFile=<path_to_truststore.jks> --trustStorePassword=<truststore_password> --trustStoreType=JKS --cipherSpec=TLS_RSA_WITH_AES_128_CBC_SHA256 --keyStoreFile=<path_to_key.jks> --keyStoreType=JKS --keyStorePassword=<keystore_password> --test

MQ without SSL:

./appConfigUtility.sh setMQConfig --appName=B2Bi --dcName=<DC2_name> --qmgr=<QMGR> --channel=<ssl_channel> --connList="<IP(PORT)>" --appQueue=<queue_name> --mqUser=<mq_username> --mqPassword=<mq_password> --adminUser=<admin_username> --adminPassword=<admin_password> --enableSSL=false --test

Note: You must modify the parameter values to suit your environment.

After running this utility, file uploads from DC2Node1 should be successful.

Note: While the application runs fine after the above steps, older inline encrypted payloads will not be accessible. However, newly uploaded inline encrypted payloads will be accessible. To access the older inline encrypted payloads, you must run the regenerateEncryptedPayloads utility as follows:

Extract Message Data:
Connect to cqlsh and run the following command to generate a messages.csv file containing all messages:
```
COPY mailbox.messages (mbx_id, msg_id, payload_type) TO 'messages.csv' WITH DELIMITER = ';' AND HEADER = FALSE;
```
Run the regenerateEncryptedPayloads Utility:
After completing your rolling upgrade to the hotfix, run the regenerateEncryptedPayloads utility on DC1Node1. This utility will convert all older inline encrypted payloads from JCEKS type to PKCS12 type, making them accessible and viewable. Use the following command:
```
./appConfigUtility.sh regenerateEncryptedPayloads --appName="B2Bi" --adminUser="<admin_username>" --adminPassword="<admin_password>" --pathtoCSVFile="<path_to_messages.csv>"
```

Upgrading to Global Mailbox v6.2.1.0

Note: Before proceeding with the upgrade to Global Mailbox 6.2.1.0, you must ensure that all older inline encrypted payloads, such as inline payloads available in the system before applying the hotfix, are accessible and viewable. Additionally, it is necessary to verify the presence of the following markers in the database as indicators of successful execution of the utilities mentioned below:

com.ibm.mailbox.inline_encrypted_payloads.converted: 'true'
com.ibm.mailbox.upgrade.keystores.converted: 'true'

Run the following query on Cassandra to check if the above two values are present:

select app_config from gatekeeper.applications where app_name='B2Bi' allow filtering;

Perform the following steps to upgrade Global Mailbox to v6.2.1.0:

Prerequisite Upgrades:
- Upgrade IIM to version 1.10.1 or higher.
- Upgrade Cassandra to version 5.2, Zookeeper to 3.9.3, and Reaper to 3.7.0.
Validation Steps in IIM:
The IIM UI will display a screen as a validation measure to ensure that the customer has read the documentation and executed the required utilities on the hotfix. By accepting this step, the customer confirms that the pre-upgrade steps have been completed as per the documentation. The same validation will trigger if the customer is upgrading via response files.
JDK17 Selection:
In IIM, on the Update Packages > Updates > Features screen, select the appropriate JDK17. By checking this box, you confirm that all pre-upgrade steps have been completed according to the documentation.
Perform a rolling upgrade of Sterling B2B Integrator + Global Mailbox nodes on DC1 and DC2