Limitations of the installation toolkit

Before using the installation toolkit to install IBM Spectrum Scale™ and deploy protocols, review the following limitations and workrounds, if any.

Table 1. Functions not supported by the installation toolkit
Function	Description	Workaround, if any
All-at-once upgrade	The installation toolkit does not support a single outage, all-at-once upgrade.	Use the manual upgrade procedure.
CES groups	The installation toolkit does not support the configuration of CES groups. This causes protocol deployments to fail in multi-network environments. This might cause issues with upgrades as well.
Clusters larger than 16 nodes	The installation toolkit does not restrict the number of nodes in a cluster in which the toolkit can be used. However, it is currently designed as a single node server so as the number of nodes increases, the bandwidth to the installer node decreases and the latency goes up. This might cause issues in implementations with more than 16 nodes. Note: If you want to use the installation toolkit in a cluster larger than 16 nodes, contact scale@us.ibm.com.
Clusters without passwordless SSH between all nodes	If clusters are setup in an `AdminCentral=True` configuration, which is a widely used configuration, the installation toolkit and protocols might not function correctly.	Set up passwordless SSH between all nodes in the cluster and to the nodes themselves using FQDN, IP address, and host name.
Compression	The installation toolkit does not configure file system compression.	After installation, configure the compression function manually. For more information, see File compression.
Concurrent upgrade	The installation toolkit does not support concurrent upgrade. You must plan for an outage depending on your setup. This is a brief outage to prevent mixed code versions from running at the same time and it is also required in case of a manual upgrade. Although the upgrade is non-concurrent, data is typically still accessible during the upgrade window. Access might be lost and need to be re-established multiple times due to how the upgrade procedure is executed among nodes.
Configuration change	The installation toolkit does not support changing existing settings and node designations. Although, it can be used to add nodes, NSDS, or file systems to an existing cluster. The installation does not support authentication reconfiguration. It does not use the authentication section of the cluster definition file during upgrade.	If you want to change the authentication method, see If you want to change the authentication method, see Modifying the authentication method.
Customer designation of sensor or collector nodes	The installation toolkit does not support customer designation of sensor or collector nodes. The installation toolkit automatically sets up sensor or collectors without allowing the user to choose which nodes will have these functions.	Use the ./spectrumscale config perfmon -d flag to disable performance monitoring and the -r flag to reconfigure performance monitoring, if required. Follow up with manual configuration of performance monitoring.
Custom profiles	The installation toolkit allows a user to choose between two GPFS™ profiles during installation prior to cluster creation.	Obtain the sample profiles that the installation toolkit uses and edit them directly. The profiles are located in the /usr/lpp/mmfs/profiles directory.
Disabling or uninstalling protocols and uninstalling GPFS	The installation toolkit does not support disabling or uninstalling protocols and uninstalling GPFS on an existing GPFS cluster.	Use the manual procedures. For information about removing exports, see Managing protocol data exports. For information about disabling protocol services, see Managing protocol services. For information about uninstalling GPFS, see Steps to permanently uninstall GPFS and/or Protocols.
Encryption	The installation toolkit does not support encrypted file systems. Therefore, installation and deployment using the installation toolkit do not work if the CES shared root file system or any other file system that the installation toolkit works with is encrypted.
EPEL and OpenStack repositories	The installation toolkit does not support the configuration of protocols when either EPEL or OpenStack repositories are configured	Remove or disable these repositories before using the installation toolkit for installation, deployment, or upgrade.
ESS awareness	The installation toolkit does not support ESS I/O or EMS nodes. Therefore, you must not add them to the installation toolkit.
File system DMAPI flag set to Yes (-z) installation and deployment	The file system DMAPI flag is used in IBM Spectrum Scale for IBM Spectrum Protect™ for Space Management and policy management, attaching with an IBM Spectrum Protect server, and with IBM Spectrum Archive™. The installation toolkit does not have any option to add a DMAPI flag to a new or existing file system. If this is desired, the installation toolkit can be used to create the file system and later the DMAPI flag can be set manually as appropriate.	For information on manual procedures for installation and deployment, see Manually installing the IBM Spectrum Scale software packages on Linux nodes.
File system DMAPI flag set to Yes (-z) upgrade	An upgrade using the installation toolkit is affected by the presence of the DMAPI flag in a few ways: If the CES shared root shares a file system that has the DMAPI flag set: In this situation, the installation toolkit cannot unmount or mount the file system on each node it attempts to upgrade unless the user has preemptively removed the DMAPI flag and stopped all DMAPI services. For more information, see Upgrading IBM Spectrum Scale on IBM Spectrum Archive Enterprise Edition (EE) nodes using the installation toolkit. This scenario is different from scenario 2, because the CES shared root file system is essential for maintaining healthy CES status of the cluster. Without the CES shared root mounted, the CES component remains unhealthy and prevents upgrade from continuing. If a non-CES shared root file system has the DMAPI flag set: In this scenario, the installation toolkit cannot unmount the file system during upgrade of each node unless the user has preemptively stopped all DMAPI related services and unmounted the file system. Following this, the installation toolkit cannot remount the file system after upgrade of each node. The file system is left unmounted and the user needs to bring it up manually on each node individually after the GPFS portion of the upgrade has finished. Otherwise, the user can wait until the entire upgrade has finished and bring up the file system on all nodes at once.	For information on using the installation toolkit for upgrade in clusters where the DMAPI flag is set to `Yes` for any of the file systems, see Upgrading IBM Spectrum Scale on IBM Spectrum Archive Enterprise Edition (EE) nodes using the installation toolkit. In the case of a failed upgrade that needs to be restarted, if some of the nodes have been restarted, then steps need to be taken to get all of the file systems mounted again and the cluster healthy before another attempt can be made. This might require stopping DMAPI services again if they auto-started after a node was upgraded.
FPO configuration for disks	The installation toolkit does not support the extra stanza file flags required for FPO setup.	Do one of the following: Create NSDs using the installation toolkit. Manually edit the NSD stanza file afterwards. The installation toolkit places the NSD stanza file in the /usr/lpp/mmfs directory. Use mmchnsd to do the changes. OR Create the cluster using the installation toolkit. Deploy protocols on the protocol nodes using the installation toolkit. Manually create the NSDs for the FPO setup.
Host-based SSH authentication	The installation toolkit does not support host-based SSH authentication. It supports only key-based SSH authentication.	Either set up key-based SSH authentication temporarily for use with the toolkit, or follow the manual steps in Manually installing the IBM Spectrum Scale software packages on Linux nodes.
IPv6	The installation toolkit supports only IPv4 address as input. Protocols also have this limitation.
iSCSI BLOCK	The installation toolkit does not install or configure the iSCSI BLOCK function.	Use the manual procedure. For more information, see Configuring Cluster Export Services as part of installing IBM Spectrum Scale on Red Hat Enterprise Linux 7.x systems.
Multiple CES networks	The installation toolkit does not support deployments with multiple CES networks. Attempting deployment in this configuration has a high probability of failure. This is because when the CES address pool has multiple subnets, the command for adding CES address assigns an IP to a CES node that cannot handle that subnet of address, which causes the deployment failure.
Multiple clusters	The installation toolkit does not support multiple clusters being defined in the cluster definition.
Multi-region object deployment	For a multi-region object deployment, the installation toolkit only sets up the region number not the replication. For information about setting up multi-region object deployment, see Enabling multi-region object deployment initially.
NFS or SMB exports configuration	The installation toolkit does not configure any exports on SMB or NFS.	Use the manual procedure. For information about configuring Cluster Export Services and creating exports, see Configuring Cluster Export Services and Managing protocol data exports.
Node function addition during upgrade	The installation toolkit does not support designating node functionality during upgrade.	To add a function to the cluster or a node, designate this new function using the installation toolkit and proceed with an installation or a deployment. Perform this action either before or after an upgrade.
NSD balancing with pools or failure groups	The installation toolkit NSD balance function only balances NSDs evenly across primary and secondary NSD servers. It does not take into account pools or failure groups.	Add nodes using the ./spectrumscale nsd add command that have the required balanced values already configured manually across failure groups, pools, and primary and secondary NSD servers.
NSD SAN attachment during initial installation	The installation toolkit cannot be used for NSD SAN attachment during initial installation because when adding NSDs using the installation toolkit, a primary and an optional comma-separated list of secondary NSD servers must be designated.	Create NSDs using the installation toolkit. Manually edit the NSD stanza file afterwards. Use mmchnsd to do the changes.
NTP server setup	The installation toolkit can be used to configure NTP clients on supported operating systems but not NTP servers.
Package managers other than `yum` or `zypper`	The installation toolkit requires the use of `yum` (RHEL) and `zypper` (SLES) package managers to function.
PPC and x86_64 mix	The installation toolkit does not support mixed CPU architecture configurations.	Use the installation toolkit on a subset of nodes that are supported and then manually install, deploy, or upgrade on the remaining nodes.
PPC LE and PPC BE mix	The installation toolkit does not support mixed endian configurations.	Use the installation toolkit on a subset of nodes that are supported and then manually install, deploy, or upgrade on the remaining nodes.
Quorum or manager configuration after cluster installation	The installation toolkit allows a user to add `-m` (to specify a manager node) and `-q` (to specify a quorum node) flags to various nodes as they are added. If the proposed configuration does not match the existing configuration, the installation toolkit does nothing to change it.	Manually change node roles by using the mmchnode command.
Remote mounted file systems	Object protocol deployment using the installation toolkit fails in case of remotely mounted file systems. This occurs because the Object component must both list and create filesets, which is not allowed on a remotely mounted file system. NFS and SMB deployments on remote mounted file systems work using the installation toolkit. The set up of a CES shared root file system when the file system is remotely mounted works using the installation toolkit.
Repository proxy	The installation toolkit does not support proxy setups when working with repositories. `yum repolist` must not have any failed repos and it must be clean.	Ensure that there are no stale or failed repositories and that only the base OS repositories are enabled during any installation toolkit activities such as installation, deployment, or upgrade.
RPMs that place dependencies upon GPFS RPMs and GPFS settings	In an environment where some RPMs have dependencies on base GPFS RPMs or GPFS settings, the installation toolkit cannot be used for installation or upgrade.
Running mmchconfig release=LATEST to complete an upgrade	The installation toolkit does not run mmchconfig release=LATEST after an upgrade. This is to give users time to verify an upgrade success and decide if the code level upgrade should be finalized.	Use mmchconfig release=LATEST after an upgrade using the installation toolkit to finalize the upgrade across the cluster.
Separate admin and daemon network	The installation toolkit does not support separate admin and daemon network.
Sudo user	The installation toolkit does not function correctly unless run as root. Running as sudo or as another user does not work.
Support for AIX®, Ubuntu, Debian, PowerKVM, Windows, Linux on Z	The installation toolkit does not support AIX, Ubuntu, Debian, PowerKVM, Windows, or Linux on Z operating systems. If these operating systems are installed on any cluster nodes, do not add these nodes to the installation toolkit.	Use the installation toolkit on a subset of nodes that are supported and then manually perform installation, deployment, or upgrade on the remaining nodes. For information about manual upgrade, see Migration, coexistence and compatibility.
Tie-Breaker NSD configuration	The installation toolkit does not configure tie-breaker disks.	Manually set the tie-breaker configuration as required using mmchconfig after completing installation using the toolkit.
Transparent cloud tiering	The installation toolkit does not install, configure, or upgrade Transparent cloud tiering.	Use the manual procedures. For more information, see Installing cloud services on IBM Spectrum Scale nodes.
Unique NSD device configuration	The installation toolkit relies upon a user having already configured and run the nsddevices sample script provided within a GPFS installation. The mmcrnsd and mmchnsd commands require running of the nsddevices script beforehand. Therefore, the installation toolkit will fail if this has not been done by the user.
Upgrade in clusters in which the entire protocol stack is not installed	The installation toolkit does not support upgrade in clusters in which the entire protocols stack was not originally installed.	Use the manual upgrade procedure. For information about manual upgrade, see Migration, coexistence and compatibility.
Upgrade while skipping over versions	The installation toolkit does not support skipping over major or minor versions of IBM Spectrum Scale releases when upgrading. For example, If an IBM Spectrum Scale cluster is at version 4.1.1.x, you cannot use the installation toolkit to upgrade it directly to version 4.2.x.	Do an initial upgrade from version 4.1.1.x to 4.2.x and then subsequently do another upgrade from 4.2.x to 4.2.x.