Installing IBM Spectrum Conductor to a shared environment

When you install to a shared environment, you install IBM Spectrum Conductor once on a shared file system (such as IBM Spectrum Scale) that every host in the cluster shares and to which every host has access.

Before you begin

Install the shared file system in your environment. For shared storage, you can use IBM Spectrum Scale. For a list of supported file systems, see Supported file systems.
If you are going to set up a shared directory for failover that uses a different location or file system, then you have to manually create an environment file so that you can source the compute host environment separately.
IBM Spectrum Conductor that is installed to a shared environment does not support a mixed cluster that uses both Linux® and Linux on POWER.
Review the requirements for installing IBM Spectrum Conductor in Installing on a compute host.
Install cURL 7.28 or higher for REST and Elastic Stack on all management hosts, and all hosts that will be used to run notebooks.
Install OpenSSL 1.0.1 or higher on all your hosts. Use the openssl version command to check the OpenSSL version.
If OpenSSL is bundled with your cURL installation, the bundled OpenSSL version must be 1.0.1 or higher and the TLS version supported with OpenSSL must include TLSv1.2. Use the curl --version command to check the OpenSSL version and the curl --help command to check the TLS version.
(Optional) The ServiceDirector and WebServiceGateway services are, by default, set to start manually. If you plan to use these services on a management host, you must manually start them after installation. To start the ServiceDirector service, the management host must use glibc version 2.14 or higher.

About this task

Follow these steps to install IBM Spectrum Conductor to a shared file system. For this installation, you install IBM Spectrum Conductor once on a host and configure the setup for the different hosts in your cluster.

For production clusters, log in with root or sudo to root permission. For evaluation clusters you can install as any user, who becomes the cluster administrator. If you install as non root, every execution user that is specified must be the Cluster Administrator.

Procedure

Define the cluster properties by setting the following environment variables. If you do not set the optional environment variables, the default values are used.

Option	Description
BASEPORT	Optional. Set for the cluster. The cluster uses seven consecutive ports from the base port. The default port number is `7869`. For example: `export BASEPORT=14899` Note: Before installation, make sure that the seven consecutive ports are not in use.
CLUSTERADMIN	Mandatory if you are installing as root. Set to any valid operating user account, which then owns all installation files. For example: `export CLUSTERADMIN=egoadmin` Note: You must create the `egoadmin` user if it does not already exist. When you set up users on all your hosts (both management and compute hosts), the execution user must use the same user ID (UID) and GID on all the hosts.
CLUSTERNAME	Optional. Set to the name of the cluster. The default is `cluster1`. For example: `export CLUSTERNAME=cluster123` Important: You cannot change the cluster name after installation.
IBM_SPECTRUM_CONDUCTOR_LICENSE_ACCEPT	Mandatory if using quiet installation mode to accept the license agreement. For example: `export IBM_SPECTRUM_CONDUCTOR_LICENSE_ACCEPT=Y`
SHARED_FS_INSTALL	Mandatory when you install to a shared file system: `export SHARED_FS_INSTALL=Y`

Optional: If you want to modify the Elastic Stack configuration, set the following environment variables. If you do not set these environment variables, the default values are used. You cannot change these values after installation.

Option	Description
ELASTIC_HARVEST_LOCATION	Specifies the directory under which the new logging structure logs' symbolic links that point to the real Spark log directory ($SPARK_HOME/logs) are stored that Filebeat uses to harvest information and support queries. All instance group logs are organized under one directory with human readable names and structure. The ELASTIC_HARVEST_LOCATION directory must be shared if IBM Spectrum Conductor is installed to a shared environment, and the value must be different from ELK_HARVEST_LOCATION. The default directory is ${EGO_TOP}/elastic_logs.
ELK_ESHTTP_PORT	Specifies the port that is used by the indexer service for communication with Elasticsearch client node, and on which the Elasticsearch RESTful APIs are accessible. The default port number is 9200.
ELK_ESHTTP_MASTER_PORT	Species the port that is used for communication to the Elasticsearch primary node. The default port number is 9201.
ELK_ESHTTP_DATA_PORT	Species the port that is used for communication to the Elasticsearch data node. The default port number is 9202.
ELK_ESSYSCOMM_PORT	Specifies the port that is used for communication to the Elasticsearch client node within the Elasticsearch cluster. The default port number is 9300.
ELK_ESSYSCOMM_MASTER_PORT	Specifies the port that is used for communication to the Elasticsearch primary node within the Elasticsearch cluster. The default port number is 9301.
ELK_ESSYSCOMM_DATA_PORT	Specifies the port that is used for communication to the Elasticsearch data node within the Elasticsearch cluster. The default port number is 9302.
ELK_LOGSHIPPER_PORT	Specifies the port that is used by the indexer service. The default port number is 5043.
ELK_HARVEST_LOCATION	Specifies the directory under which the old logging structure logs are stored that Filebeat uses to harvest information and support queries. The default directory is /var/tmp/elk_logs/.
ELASTICSEARCH_DATA_LOCATION	Specifies the Elasticsearch data directory. The default value for this directory can be overwritten during installation, or is configurable at $EGO_CONFDIR/../../integration/elk/conf/elk.conf. Set this value during installation and do not change it afterwards.
LOGSTASH_LOCATION	Specifies the Logstash data directory. The default value for this directory can be overwritten during installation, or is configurable at $EGO_CONFDIR/../../integration/elk/conf/elk.conf. Set this value during installation and do not change it afterwards.
FILEBEAT_LOCATION	Specifies the Filebeat data directory. The default value for this directory can be overwritten during installation, or is configurable at $EGO_CONFDIR/../../integration/elk/conf/elk.conf. Set this value during installation and do not change it afterwards.

Optional: If you want to support instance group deployments to root squashed shared file systems, complete these configurations:
Note:
- Within a IBM Spectrum Conductor environment, Docker does not support root squashed installations and root squash instance group deployments.
- If you do not specify export ROOT_SQUASH_INSTALL=Y during this installation step, you can manually migrate IBM Spectrum Conductor to use root squashed shared file systems after installing and configuring IBM Spectrum Conductor.
- For NFS systems, the NFS share should not be root squashed.
1. Set the ROOT_SQUASH_INSTALL environment variable:
```
export ROOT_SQUASH_INSTALL=Y
```
2. Configure the cluster administrator OS user to be a member of the administrator OS user group of every instance group administrator OS user. (By default, the administrator OS user group of an instance group is the primary group of the instance group administrator OS user; or alternatively, the administrator OS user group can be provided as a configuration parameter when creating an instance group.) You can complete this configuration using your operating system's tools for managing OS users and their properties.
3. Ensure correct permissions for the EGO log and work directories required by IBM Spectrum Conductor: set them to locations that are not root squash enabled, using the EGO_WORKDIR and EGO_LOGDIR parameters in the ego.conf configuration file on each host in your cluster. For example:
```
# EGO working and logging directory
EGO_WORKDIR=/myworkdirectory/work
EGO_LOGDIR=/mylogdirectory/log
```
  Once you complete this change to the ego.conf file on each host, restart EGO on all hosts in the cluster:
```
egosh ego restart all
```
Optional: If you do not want to enable SSL communication for your web servers and instance groups (enabled by default), set the DISABLESSL environment variable:
```
export DISABLESSL=Y
```
Check for port conflicts to ensure that the web servers ports are free. The web servers are accessible on the following default ports:

Web server With SSL Without SSL

Web server for the cluster management console 8443 8080

REST web server 8543 8180

ascd web server 8643 8280

Important: You must use the same SSL setting for the cluster management console and the RESTful web servers. If you disable SSL for one, you must disable SSL for the other as well. This setting also takes effect for cloud bursting with host factory. Ensure that SSL for all these functions is configured consistently in the cluster; without a uniform configuration, errors occur. Note, however, that when SSL is uniformly enabled, you can use different certificates and keys as required.
Optional: If you want to use the non-production Derby database for reporting, set the DERBY_DB_HOST environment variable:
```
export DERBY_DB_HOST=primary_host
```
where primary_host is the primary host or another management host that serves as the database host.
You cannot use the Derby database for production clusters. To produce regular reports for a production cluster, you must configure an external production database after installation. See Setting up an external database for production Setting up an external database for production.

Without the Derby database or an external database, you cannot generate reports or view the Rack View on the cluster management console. If you do not require these functions, you can manually disable individual data loaders after installation. See Data loaders Data loaders.
For Ubuntu installation, run the apt-get install rpm in Ubuntu to install the RPM utility.
Run the IBM Spectrum Conductor installer package.
- To install with default settings, enter the following commands:
  Note: IBM Spectrum Conductor is installed in its default directory: /opt/ibm/spectrumcomputing. This directory must be available on a shared file system, as specified in the install command.
  Entitled version:
```
export IBM_SPECTRUM_CONDUCTOR_LICENSE_ACCEPT="Y"
conductor2.4.1.0_ppc64le.bin --quiet
```
  Evaluation version:
```
export IBM_SPECTRUM_CONDUCTOR_LICENSE_ACCEPT="Y"
conductoreval2.4.1.0_ppc64le.bin --quiet
```
  - For Linux 64-bit:
```
conductor2.4.1.0_x86_64.bin
```
  - For Linux on POWER® LE:
```
conductor2.4.1.0_ppc64le.bin
```
  Note: Alternative method: If you must use .rpm files instead of .bin, extract the .rpm files, install the ego*.rpm files, then install the ascd*.rpm and conductorspark*.rpm files.
  For example, first extract the .rpm files from the conductor2.4.1.0_x86_64.bin or conductor2.4.1.0_ppc64le.bin package by running:
```
conductor2.4.1.0_x86_64.bin --extract extract_directory
```
```
conductor2.4.1.0_ppc64le.bin --extract extract_directory
```
  where extract_directory specifies the directory to extract .rpm files.
  Next, install each .rpm file in order by running:
```
rpm -ivh rpm_file_name
```
  For example:
```
rpm -ivh egocore-version.architecture.rpm
```
```
rpm -ivh egocore-3.8.0.0.rpm
```
  Additional note for Ubuntu: To install the .rpm files on Ubuntu, you must also specify the --ignorearch parameter. For example:
```
rpm -ivh --ignorearch rpm_file_name
```
- To install to a custom location, enter the following commands:
  Entitled version:
```
export IBM_SPECTRUM_CONDUCTOR_LICENSE_ACCEPT="Y"
conductor2.4.1.0_ppc64le.bin --prefix install_location --dbpath dbpath_location --quiet
```
  Evaluation version:
```
export IBM_SPECTRUM_CONDUCTOR_LICENSE_ACCEPT="Y"
conductoreval2.4.1.0_ppc64le.bin --prefix install_location --dbpath dbpath_location --quiet
```
  - For Linux 64-bit:
```
conductor2.4.1.0_x86_64.bin --prefix install_location --dbpath dbpath_location
```
  - For Linux on POWER LE:
```
conductor2.4.1.0_ppc64le.bin --prefix install_location --dbpath dbpath_location
```
  where:
  - --prefix install_location specifies the absolute path to the installation directory. Specifying the installation path with the --prefix parameter is mandatory when installing on a shared file system, unless the default directory /opt/ibm/spectrumcomputing is already mounted from a shared file system. If you install without the --prefix option, IBM Spectrum Conductor is installed in its default directory: /opt/ibm/spectrumcomputing. Ensure that the path is set to a clean directory.
  - --dbpath dbpath_location sets the RPM database to a directory different from the default /var/lib/rpm. The --dbpath parameter is optional.
  For example:
```
./conductor2.4.1.0_x86_64.bin --prefix  /gpfs/test/platform4 --dbpath /gpfs/test/platform4/db
```
```
./conductor2.4.1.0_ppc64le.bin --prefix  /gpfs/test/platform4 --dbpath /gpfs/test/platform4/db
```
  Note: Alternative method: If you must use .rpm files instead of .bin, extract the .rpm files, install the ego*.rpm files, then install the ascd*.rpm and conductorspark*.rpm files.
  For example, first extract the .rpm files from the conductor2.4.1.0_x86_64.bin or conductor2.4.1.0_ppc64le.bin package by running:
```
conductor2.4.1.0_x86_64.bin --extract extract_directory
```
```
conductor2.4.1.0_ppc64le.bin --extract extract_directory
```
  where extract_directory specifies the directory to extract .rpm files.
  Next, install each .rpm file in order by running:
```
rpm -ivh --prefix install_location --dbpath dbpath_location rpm_file_name
```
  For example:
```
rpm -ivh --prefix /opt/mydir --dbpath /opt/mydir/mydb egocore-3.8.0.0egocore-version.architecture.rpm
```
  Additional note for Ubuntu: To install the .rpm files on Ubuntu, you must also specify the --ignorearch parameter. For example:
```
rpm -ivh --ignorearch --prefix install_location --dbpath dbpath_location rpm_file_name
```
- To install without user interaction, enter one of the following commands:enter the following command:
```
conductor2.4.1.0_ppc64le.bin --quiet
```
  - For Linux 64-bit:
```
conductor2.4.1.0_x86_64.bin --quiet
```
  - For Linux on POWER LE:
```
conductor2.4.1.0_ppc64le.bin --quiet
```
  where --quiet suppresses prompts during installation.
After installation is complete, source the environment:
- (csh) source $EGO_TOP/cshrc.platform
- (bash) . $EGO_TOP/profile.platform
where $EGO_TOP is the path to your installation directory (the default path is /opt/ibm/spectrumcomputing).

IBM Spectrum Conductor automatically creates this profile.platform (or cshrc.platform file when you use CSH) on management hosts during installation. The profile.platform (cshrc.platform) file sources other files, all of which together set the environment for management hosts in the cluster.
Create the files that set up the compute host environment in your cluster.
Default: files are automatically created
IBM Spectrum Conductor automatically creates the profile.platform (or cshrc.platform file when you use CSH) on management hosts during installation. The profile.platform (cshrc.platform) file sources other files, all of which together set the environment for management hosts in the cluster.
Use the profile.platform (cshrc.platform) file to set your environment, as follows:

(csh) source $EGO_TOP/cshrc.platform

(bash) . $EGO_TOP/profile.platform
Manually create the files
If you are going to set up a shared directory for failover that uses a different location or file system, then you have to manually create the profile.platform.comp (or cshrc.platform.comp file when you use CSH) on compute hosts. The profile.platform.comp (cshrc.platform.comp) file, along with other files, sets the environment for compute hosts in the cluster. To create this file and set up your environment, complete these steps:
1. Copy the profile.platform file to a new profile.platform.comp file:
  cp profile.platform profile.platform.comp
2. Copy the $EGO_TOP/kernel/conf/profile.ego file to a new $EGO_TOP/kernel/conf/profile.ego.comp file:
  cp $EGO_TOP/kernel/conf/profile.ego $EGO_TOP/kernel/conf/profile.ego.comp
3. Open the profile.platform.comp file, locate the line with profile.ego in it, and add the .comp file extension to the profile.ego file.
  For example, change this line:
  /opt/ibm/spectrumcomputing/kernel/conf/profile.ego
  to this:
  /opt/ibm/spectrumcomputing/kernel/conf/profile.ego.comp
4. Source your compute host environment:
  
  (csh) source $EGO_TOP/cshrc.platform.comp
  
  (bash) . $EGO_TOP/profile.platform.comp
Link the Elastic Stack harvesting directory to a unique directory on your shared file system. This harvesting directory is the value of the ELK_HARVEST_LOCATION environment variable; by default, /var/tmp/elk_logs/.
1. As the cluster administrator, create a unique directory in your shared file system for each host included in your cluster. For example, to create directories for hostA and hostB in your IBM Spectrum Scale file system, enter:
```
mkdir /gpfs/conductor/var/tmp/elks_logs/hostA 
mkdir /gpfs/conductor/var/tmp/elks_logs/hostB
```
2. Determine your cluster administrator group:
```
id CLUSTERADMIN -g
```
  where CLUSTERADMIN is your cluster administrator account (for example, egoadmin).
3. Ensure correct permissions for each directory that you created in step 11.a:
```
chown -Rh $CLUSTERADMIN:$ADMINGROUP $DIRECTORY
chmod u+rwx,g+rwxs,o+rwx $DIRECTORY
```
  where:
  - $CLUSTERADMIN is your cluster administrator account (for example, egoadmin).
  - $ADMINGROUP is the operating system group for your cluster administrator (determined in step 11.b).
  - $DIRECTORY is the host's unique directory in your shared file system (created in step 11.a).
4. On each host in the cluster, check if the directory defined by ELK_HARVEST_LOCATION exists. If it does, remove the directory (default /var/tmp/elk_logs).
5. On each host, create a link to the host's harvesting location in your shared file system:
```
ln -s DIRECTORY ELK_HARVEST_LOCATION
```
  where:
  - DIRECTORY is the host's unique directory in the shared file system, created in step 11.a.
  - ELK_HARVEST_LOCATION is the directory that is specified by the ELK_HARVEST_LOCATION environment variable (default is /var/tmp/elk_logs).
  Note: In RHEL 7.1, symbolic link protection is enabled by default. You must either create the symbolic link as root, or disable symbolic link protection. For more information, see Protecting Hard and Symbolic Links.

Web server	With SSL	Without SSL
Web server for the cluster management console	8443	8080
REST web server	8543	8180
ascd web server	8643	8280

Optional: If you plan on scheduling batch applications, you can update the default algorithm and key size that is used to encrypt the scheduling user's token. The scheduling user token is maintained for all users who schedule batch applications.

Note: This configuration can be updated only before the cluster is started for the first time. If you do not change the parameters, the default values are used.

Open the ascd.conf configuration file at $EGO_CONFDIR/../../ascd/conf.

Edit both parameters as required:

Option	Description
CONDUCTOR_SPARK_SCHEDULED_APP_CIPHER_ALGORITHM	Specifies the algorithm that is used to encrypt the token for the scheduling user. Valid values are AES (default) or DESede.
CONDUCTOR_SPARK_SCHEDULED_APP_CIPHER_KEYSIZE	Specifies the key size that is used to encrypt the token for the scheduling user. Valid values are as follows: If CONDUCTOR_SPARK_SCHEDULED_APP_CIPHER_ALGORITHM=AES, set the key size to 128 (default), 192, or 256 bits. If CONDUCTOR_SPARK_SCHEDULED_APP_CIPHER_ALGORITHM=DESede, set the key size to 112 or 168 bits.

Save your changes.

What to do next

After you install IBM Spectrum Conductor, join the host and set entitlement. See Entitling IBM Spectrum Conductor.
For a root squash installation, after you install, ensure you also change the EGO log and work directories to locations that are not root squash enabled, as described in step 4.c.