Completing the upgrade to a new level of IBM Storage Scale

It is a good idea to use the cluster for a while with the new level of IBM Storage Scale installed, until you are sure that you are ready to permanently upgrade the cluster to the new level.

When you are ready to permanently upgrade the cluster, follow the steps in this topic to complete the upgrade. If you decide not to complete the upgrade, you can revert to the previous level of IBM Storage Scale. For more information, see Reverting to the previous level of IBM Storage Scale.

Before you begin this task, verify that you have upgraded all the nodes in the cluster to the latest licensed version of IBM Storage Scale.

When you run mmchconfig release=LATEST in Step 2 of these directions, you can add other parameters and their values to the command line:

mmchconfig release=LATEST,<parameterN=value>,<parameterN+1=value>...

The following table describes the options and parameters that are referred to in this topic.

Table 1. Other options and parameters with mmchconfig release=LATEST
Option/Parameter	Purpose	Comment
--accept-empty-cipherlist-security	Specify this option if you want to continue running the cluster with the lowest level of security for communications between nodes or with other clusters. That is, you want the cipherList attribute to remain set to EMPTY or undefined. For more information, see Security mode.	It is a good idea to have some level of security for cluster communications: Set cipherList to AUTHONLY or to a supported cipher: `mmchconfig cipherList=AUTHONLY` Do not include the parameter --accept-empty-cipherlist-security when you run `mmchconfig release=LATEST.` For more information, see Security mode.
--accept-no-compliance-to-nist-standards	Specify this option if you do not want to continue running the cluster with the security transport that follows the NIST SP800-131A recommendations. For more information, see NIST compliance.	It is recommended to run the security transport that follows the NIST SP800-131A recommendations: Set nistCompliance attribute to SP800-131A `mmchconfig nistCompliance=SP800-131A` Set nistCompliance attribute to SP800-131A at the same time when you upgrade IBM Storage Scale to a new level. `mmchconfig release=LATEST,nistCompliance=SP800-131A`
mmfsLogTimeStampISO8601={yes \| no}	Setting this parameter to no allows the cluster to continue running with the earlier log time stamp format. For more information, see Security mode.	Set mmfsLogTimeStampISO8061 to no if you save log information and you are not yet ready to switch to the new log time stamp format. After you complete the upgrade, you can change the log time stamp format at any time with the mmchconfig command. Omit this parameter if you are ready to switch to the new format. The default value is yes
tscCmdAllowRemoteConnections	Setting this attribute to no forces the ts* commands to communicate with the local mmfsd daemon only over a UNIX domain socket (UDS), and to communicate with mmfsd daemons running on other nodes over the RPC communication framework used by IBM Storage Scale.	After a cluster is upgraded, the tscCmdAllowRemoteConnections parameter value is set to yes by default. Setting the tscCmdAllowRemoteConnections configuration to no requires a cluster minimum release level of 5.1.3 or later. System administrators can set the tscCmdAllowRemoteConnections value to no after setting the minReleaseLevel=LATEST, which is 5.1.3 or later. Setting tscCmdAllowRemoteConnections to no in a multi-cluster setting must be done after ensuring that the remote clusters are running IBM Storage Scale version 5.1.3 or later. For more information, see mmchconfig command.

Note: It is not recommended to use the --accept-no-compliance-to-nist-standards option and this option might not be available in the subsequent releases.

Verify that the SHA message digest and the cipherList configuration variable are set to valid values.
Note:
- The SHA message digest is a hash result that is generated by a cryptographic hash function.
- The cipherList variable specifies the security mode for communications among nodes in the cluster and with nodes in other clusters. For more information, see mmchconfig command.
Follow these steps:
1. Display the current values by entering the following command. The listing shows both the command and example output:
```
# mmauth show .
```
  Cluster name: zounds.cluster (this cluster) Cipher list: (none specified) SHA digest: (undefined) File system access: (all rw)
2. If the value for the SHA digest is (undefined), follow these steps:
  1. Enter the following command to generate a public/private key pair and an SHA message digest:
```
mmauth genkey new
```
  2. Enter mmauth show . again and verify that the value for SHA digest is no longer (undefined).
3. If the value for cipherList is (none specified) or EMPTY, do one of the following actions:
  - If you want a level of security in communications between nodes and with other clusters, follow these steps:
    1. Set cipherList to AUTHONLY or to a supported cipher:
      mmauth update . -l AUTHONLY
    2. Enter mmauth show . again and verify that the value for cipherList is no longer (none specified) or EMPTY.
  - If you do not want a level of security cluster communications, let cipherList remain set to (none specified) or EMPTY.
Enter the following command to upgrade the cluster configuration data and enable new functionality. You can add the parameters that are described in Table 1 to the command line:
```
mmchconfig release=LATEST 
```
Note: Until you run the mmchconfig release=LATEST command, the management GUI might not be fully operational at the new code level.

Important: If the mmchconfig command detects any nodes that are not available or cannot be reached, it lists the names of those nodes. If any such nodes are listed, correct the problem and run the command again until it verifies all the nodes and completes successfully.
Note: If the mmchconfig command fails with an error message that indicates that cipherlist is set to EMPTY, do one of the following actions:
- If you want the cluster to run with a higher security mode than EMPTY, set cipherList to AUTHONLY or to a supported cipher:
```
mmauth update . -l AUTHONLY
```
  Return to the first part of Step 2 and run the mmchconfig command as before.
- If you want the cluster to continue with the security mode set to EMPTY, return to the first part of Step 2 and run the mmchconfig command with the additional parameter --accept-empty-cipherlist-security.
If you have not already done so, assign an appropriate IBM Storage Scale license to each of the nodes in the cluster.
See IBM Storage Scale license designation for a detailed description of the IBM Storage Scale license types. To see what the minimum required IBM Storage Scale license is for each of the nodes in the cluster, enter the following command:
```
mmlslicense -L
```
To assign an IBM Storage Scale server license to the nodes that require it, enter the following command:
```
mmchlicense server -N NodeList
```
To assign an IBM Storage Scale client license to the nodes that require it, enter:
```
mmchlicense client -N NodeList
```
Enable backward compatible format changes or upgrade all file systems to the latest metadata format changes.
Attention: Before you continue with this step, it is important to understand the differences between mmchfs -V compat and mmchfs -V full:
- If you enter mmchfs -V compat, only changes that are backward compatible with the previous file system version are enabled. Nodes in remote clusters that are running at the previous IBM Storage Scale version or later will still be able to mount the file system. Nodes running an IBM Storage Scale version earlier than the previous version will no longer be able to mount the file system. For example, if you are upgrading from IBM Storage Scale 5.0.2.x to 5.1.0.x, a file system of version corresponding to IBM Storage Scale 5.0.2.x will remain mountable from a remote cluster running IBM Storage Scale 5.0.2.x or later. However, a file system of version corresponding to IBM Storage Scale 5.0.0.0 will remain mountable from a remote cluster running IBM Storage Scale 5.0.0.0 or later.
  Running in the compatibility mode might prevent enablement of new functionality that relies on disk structures available only with the latest version.
- If you enter mmchfs -V full, all new functions that require different on-disk data structures are enabled. Nodes in remote clusters that run an older IBM Storage Scale version will no longer be able to mount the file system. For example, remote container native clusters. If any nodes that run an older IBM Storage Scale version have mounted the file system at the time this command is entered, the mmchfs command fails. This consideration might also apply to minor releases within the same major release. For example, release 5.0.2.x and release 5.1.0.x might have different metadata formats.
To enable backward-compatible format changes, enter the following command:
```
mmchfs FileSystem -V compat
```
To upgrade the desired file system to the latest metadata format changes, enter the following command:
```
mmchfs FileSystem -V full
```
Certain new file system features might require more processing that cannot be handled by the mmchfs -V command alone. To fully activate such features, in addition to mmchfs -V, you must also run the mmmigratefs command.

Note: The first mount of a file system after you run mmchfs -V might fail with a no-disk-space error. This situation might occur if the file system is relatively low on metadata disk space (10% or less free). If so, enter the command again. Typically the file system is mounted without a problem after the initial failed mount.

After completing the upgrade, you might need to enable cluster configuration repository (CCR) and fast extended attributes (fastea).

Enable cluster configuration repository (CCR) and fast extended attributes (fastea) as follows, if required.
1. Enable the CCR in the cluster as follows.
```
mmchcluster --ccr-enable
```
2. Enable fastea in the cluster as follows.
  1. Check if fastea is enabled.
```
mmlsfs FileSystemName --fastea
```
    A sample output is as follows.
    flag value description ------------------- ------------------------ ----------------------------------- --fastea Yes Fast external attributes enabled?
  2. If fastea is not enabled, issue the following command to enable it.
```
mmmigratefs FileSystemName --fastea
```
    For more information, see mmmigratefs command.
  Note: The following features are a few among several IBM Storage Scale features that require fast extended attributes (fastea) to be enabled to work.
  - Clones
  - Independent filesets
  - Encryption
  - Active file management (AFM)
If you have file audit logging or clustered watch folder enabled on any file systems, follow the steps in Upgrade paths and commands for file audit logging and clustered watch folder.

Note: Ensure that you include the gpfs.librdkafka package as part of the upgrade process if those packages are currently installed.
Set the value of the tscCmdAllowRemoteConnections attribute to no.
The tscCmdAllowRemoteConnections attribute specifies whether the ts* commands are allowed to use the remote TCP/IP connections when communicating with the local or other mmfsd daemons. The scope of the tscCmdAllowRemoteConnections attribute is local to a cluster and it has the same value on all the nodes in the cluster.
Warning: Make sure that all remote clusters must be at 5.1.3 or later. Setting the tscCmdAllowRemoteConnections parameter to no impacts the remote cluster functions, hence the recommendation is to check the version of the remote clusters. If any remote cluster is running a version of IBM Storage Scale older than 5.1.3, setting the tscCmdAllowRemoteConnections parameter value to no in the home cluster can generate Operation not permitted errors. For more information, see mmchconfig command.