Upgrading on a shared file system

Follow these steps for a rolling upgrade when your existing cluster is installed to a shared file system. Installing on a shared file system is supported on Linux.

Before you begin

Ensure that you have read the prerequisites in Before you perform a rolling upgrade.
Similar to when you installed your previous version of IBM Spectrum Symphony, you can use one shared location for all hosts, and this scenario is fully supported by the product. You can also have more than one shared locations (for example, a separate share for management hosts and another for compute hosts). For this scenario, the host configuration and work is redirected to the appropriate shared location; you manually create an environment file so that you can source the compute host environment separately.
You require the IBM Spectrum Symphony installation package and the appropriate product edition entitlement file (sym_*_entitlement.dat).

Procedure

Log on to your primary host as root, or as the cluster administrator with root access (set by running the egosetsudoers.sh command to grant root privileges to the cluster administrator).
Specify the entitlement file directory and file name. For example:
```
export ENTITLEMENTFILE=/opt/temp/sym_adv_entitlement.dat
```
Install the latest version of IBM Spectrum Symphony on your primary host according to the steps for Linux® and Linux on POWER® in Installing to a shared environment.
You do not need to set any environment variables (other than the ENTITLEMENTFILE environment variable that you already specified in step 2). The rolling upgrade process uses the parameter values, and installation location, from your previous installation.
Once complete:
1. Source the IBM Spectrum Symphony environment:
  - (csh) source $EGO_TOP/cshrc.platform
  - (bash) . $EGO_TOP/profile.platform
2. Restart EGO on your local host:
```
egosh ego shutdown
egosh ego start
```
Complete this step only if you used the alternative method and installed the latest version of IBM Spectrum Symphony by using .rpm files instead of the .bin package. In this case, after you install all the applicable .rpm files:
1. Source the IBM Spectrum Symphony environment:
  - (csh) source $EGO_TOP/cshrc.platform
  - (bash) . $EGO_TOP/profile.platform
2. Source the upgrade environment:
  - (csh) source $EGO_TOP/upgrade/conf/cshrc.upgrade
  - (bash) . $EGO_TOP/upgrade/conf/profile.upgrade
3. Run the egoupgrade host command on the local host:
```
egoupgrade host [-f]
```
  For syntax usage and details, see egoupgrade.
Log on to each of your primary candidate hosts as the same user as in step 1, and complete these steps:
1. Source the IBM Spectrum Symphony environment:
  - (csh) source $EGO_TOP/cshrc.platform
  - (bash) . $EGO_TOP/profile.platform
2. Restart EGO on the host:
```
egosh ego shutdown
egosh ego start
```
IBM Spectrum Symphony automatically creates this profile.platform (or cshrc.platform file when you use CSH) on management hosts during installation. The profile.platform (cshrc.platform) file sources other files, all of which together set the environment for management hosts in the cluster.
Log on to each of your management hosts as the same user as in step 1, and repeat step 5 on each host.

Note: Ensure that you specify the same installation path on the primary candidate hosts as you used for the primary host installation; otherwise, the installer will not be able to start and stop the primary candidate host, causing issues for the cluster upgrade process.
Log on to each of your compute hosts as the same user as in step 1, and complete these steps:
One shared location
IBM Spectrum Symphony automatically creates the profile.platform (or cshrc.platform when you use CSH) file on management hosts during installation. The profile.platform (cshrc.platform) file sources other files, all of which together set the environment for hosts in the cluster. If you use one shared location for all hosts, use the profile.platform (cshrc.platform) file to set your environment, as follows:
1. Source the environment:
  
  (csh) source $EGO_TOP/cshrc.platform
  
  (bash) . $EGO_TOP/profile.platform
2. Restart EGO on the host:
  egosh ego shutdown egosh ego start
More than one shared location
If you have more than one shared locations (for example, a separate share for management hosts and another for compute hosts), you manually created the profile.platform.comp (or cshrc.platform.comp when you use CSH) file on compute hosts. After you upgrade IBM Spectrum Symphony, you must also edit the profile.platform.comp (cshrc.platform.comp) file to the use the upgraded product version, and then source the environment, as follows:
1. Open each of the files in profile.platform.comp and update the previous EGO_Version to the current one.
2. Source the IBM Spectrum Symphony environment:
  
  (csh) source $EGO_TOP/cshrc.platform.comp
  
  (bash) . $EGO_TOP/profile.platform.comp
3. Restart EGO on the host:
  egosh ego shutdown egosh ego start
For ease of administration, you can set up automatic system startup when a host restarts. You can do this by configuring Linux run communication (rc. On a shared file system, you install IBM Spectrum Symphony once on the primary host, and if you logged on as root to run the IBM Spectrum Symphony installation, the installer resets the rc daemon to start the IBM Spectrum Symphony cluster on the primary host using the upgraded EGO binary. However, if you did not install as root, and want to use rc to automatically start up the EGO service on the upgraded version of IBM Spectrum Symphony, then run egosetrc.sh now to configure automatic startup.
If you want to also automatically start the EGO service, for the other hosts in the shared file system, if the hosts restart, you can do so with either of these methods:
- To use the rc, which uses the init daemon process, run egosetrc.sh on each host.
- To use the systemd, which uses the systemd daemon process, first run egoremoverc.sh to disable rc, and then run egosetsystemd.sh to enable systemd. This automatically starts up of the EGO service using systemd if your system restarts.
Configure the cluster for the upgrade (ensure that you complete the following set of steps together):
1. If you ran the egoconfig mghost command to configure management hosts to specify a shared directory that the hosts use to access common cluster files, then run this command again for the upgraded cluster, as follows:
  1. Log on to the primary host as the cluster administrator.
  2. Stop all cluster services managed by EGO:
```
egosh service stop all
```
  3. Stop the cluster:
```
egosh ego shutdown all
```
  4. Run the egoconfig mghost command:
```
egoconfig mghost shared_top
```
    Note: Ensure that you run this command before you upgrade your cluster in subsequent steps. Running the egoconfig mghost command after you run the egoupgrade cluster command might severely impact cluster functionality.
  5. Start the cluster:
```
egosh ego start all
```
  Backing up your cluster to a shared directory does not include any service packages: In a later step, you will run the egoupgrade cluster command, which backs up existing configuration files to a backup directory within your shared location (shared_top) directory. This backup includes configuration files in the system (such as EGO service XML files, application profile files, and so on), used for IBM Spectrum Symphony 7.3.2, but this does not include backing up the IBM Spectrum Symphony service package for your previous version. Do not upgrade the service package during your cluster upgrade and application upgrade stages. If you upgrade the service package during these stages, and you later require rolling back IBM Spectrum Symphony, the service package will not be correctly configured for the your previous version's application profile and your application will not work. If you do roll back, ensure that you have the previous version's service package working with the previous version's application profile.
2. Log on to your primary host as root or sudo to root (a user that has been set up in the egosetsudoers configuration file) if you installed IBM Spectrum Symphony as root. If you didn’t install as root, you must log on with the cluster administrator ID. If you did not already source the IBM Spectrum Symphony environment, do it now:
  - (csh) source $EGO_TOP/cshrc.platform
  - (bash) . $EGO_TOP/profile.platform
3. This step is applicable only if you are upgrading from IBM Spectrum Symphony 7.30, and you enabled the symchargebackloader data loader for the Explorer Charge Back chart (from step 4 in Enabling the Explorer reports feature for 7.3), and you will require the charge back data after upgrading to IBM Spectrum Symphony 7.3.2, then complete the following sub-steps, to export Explorer Charge Back chart indices and data from you previous version of IBM Spectrum Symphony:
  1. Stop the cluster management console (WEBGUI) and Explorer reports services on your previous version of IBM Spectrum Symphony:
```
egosh service stop WEBGUI
egosh service stop Explorer
```
  2. Export charge back indices and data from your previous version of IBM Spectrum Symphony using the Symphony_export_ChargeBack.sh script (you will import this data to your updated version of IBM Spectrum Symphony after running egoupgrade cluster in a later step):
    1. Source the environment:
      
      (csh) source $EGO_TOP/cshrc.platform
      
      (bash) . $EGO_TOP/profile.platform
    2. Change to the $EGO_TOP/integration/elk/version/scripts directory in IBM Spectrum Symphony 7.3.2:
      cd $EGO_TOP/integration/elk/version/scripts
    3. Run the export script:
      ./Symphony_export_ChargeBack.sh Admin Admin export_directory
      where export_directory is the directory where you want the script to export the charge back indices and data. The script creates a directory under this export directory. If not specified, script exports to the current_directory/new_directory directory.
      
      Once the script completes, it creates a JSON file that contain the indices and stores this data in export_directory/new_directory directory.
4. This step is applicable only if you completed the Explorer Charge Back chart indices and data exporting step (step 9c).
  Import Explorer Charge Back chart indices and data that you exported from your previous version of IBM Spectrum Symphony, to IBM Spectrum Symphony 7.3.2:
  1. Restart the cluster management console (WEBGUI) and Explorer reports services on IBM Spectrum Symphony 7.3.2:
    egosh service start WEBGUI egosh service start Explorer
  2. Import the charge back indices and data that you exported (in step 9c) from your previous version of IBM Spectrum Symphony using the Symphony_import_ChargeBack.sh script:
    
    Change to the $EGO_TOP/integration/elk/version/scripts directory in IBM Spectrum Symphony 7.3.2:
    cd $EGO_TOP/integration/elk/version/scripts
    
    Run the import script:
    ./Symphony_import_ChargeBack.shAdmin Admin export_directory
    where export_directory is the directory where you exported the charge back indices and data from the previous version of IBM Spectrum Symphony. The script takes the JSON file containing the exported information from the export_directory/new_directory directory and imports that information into IBM Spectrum Symphony 7.3.2 for Explorer reports.
  3. Verify that the script ran successfully:
    
    Verify that the script created the new indices. For example:
    curl -u Admin:Admin -k -H'Content-Type: application/json' -XGET https://host_name:port/_cat/indices?pretty
    You should see a list of all the new_indices-yyyy.mm.dd indices.
    
    Verify that data exists the indices. Run the following verification for one of the indices. For example:
    curl -u Admin:Admin -k -H'Content-Type: application/json' -XGET https://host_name:port/new_index/_search?pretty
    You should see a data for a new index.
5. Source the upgrade environment and run the cluster upgrade command:
  1. Source the upgrade environment:
    - (csh) source $EGO_TOP/upgrade/conf/cshrc.upgrade
    - (bash) . $EGO_TOP/upgrade/conf/profile.upgrade
  2. Run the cluster upgrade command:
```
egoupgrade cluster [-f] [-u user_account] [-x password]
```
    For syntax usage and details, see egoupgrade.
    Tip:
    
    While you upgrade the cluster, the system can log some failed operations, but they might not impact the upgrade process. Check the egoupgrade log under the $EGO_TOP/upgrade/logs directory to assess whether the failures affect your upgraded environment.
    
    Before you run the egoupgrade cluster command, if you need to use the IBM Spectrum Symphony CLI or submit old workload from your previous version, first source the previous IBM Spectrum Symphony environment:
    
    (csh) source $EGO_TOP/cshrc.platform.previous_version
    
    (bash) . $EGO_TOP/profile.platform.previous_version
    
    If you are upgrading from IBM Spectrum Symphony 7.3, before you run the egoupgrade cluster command, if you are unable to start the HostFactory service, modify the hostfactory.xml file and change this line:
    <ego:Command{EGO_TOP}\3.8\{EGO_TOP}\3.8\>${EGO_TOP}/${EGO_VERSION}/${EGO_MACHINE_TYPE}/etc/hostfactory</ego:Command>
    to use the EGO version from your previous IBM Spectrum Symphony environment, change the line to:
    <ego:Command>${EGO_TOP}\3.8\${EGO_MACHINE_TYPE}\etc\hostfactory</ego:Command>
Open a command prompt on the upgraded version, and run the egosh -V command to confirm the upgrade on your hosts. The output should list the EGO version as 4.0.0.
```
egosh -V
EGO 4.0.0 build xxxxxx, MMM DD YYYY
binary type: linux-x86_64
```
Access the cluster management console:
1. Start a web browser and clear the browser cache and cookies. Check your browser documentation for details.
  
  Note: All users who connect to the cluster management console must clear their browser cache and cookies before you access the console. As a cluster administrator, ensure that you notify all your users about this requirement.
2. Log in to the cluster management console. See Locating the cluster management console.
  
  If you personalized preferences for the cluster management console, recreate your preferences. For example, if you set up table preferences to include custom columns, edit the Preferences link for those pages to change the default settings.
3. If your environment includes other products in the IBM Spectrum Computing family (such as IBM Spectrum Conductor), the post-installation wizard launches. Complete the wizard to set up extra components as required.
Verify that you have installed the applicable packages:
- For Linux and Linux on POWER hosts: Access the System & Services > Software Components page within the cluster management console to see the software components installed on each host. This page indicates the installed software component packages, and active versions on each host. With an upgrade, you may have more than one version of the same software component installed, but only the upgraded version should be active.
- Query the installation packages on the host database using rpm commands:
```
rpm -qa [--dbpath dbpath_location] | grep package_name
```
  Note that if you installed IBM Spectrum Symphony using the --dbpath option, use the same dbpath_location to verify. For example, if you installed using /data/abc/rpmdb as the dbpath_location, then run the verification commands using that database location. Here is an example:
```
rpm -qa --dbpath /data/abc/rpmdb | grep soammrmgmt
```
For a list of installation package names, refer to Files within the installation packages.

Results

Your hosts are upgraded to the latest version of IBM Spectrum Symphony . Your previous IBM Spectrum Symphony configuration (including users, custom user roles, consumers, resource groups, resource plans, or custom reports; and enabled features) is also migrated to the upgraded product version.

What to do next

After you upgrade each host and the cluster to the latest version of IBM Spectrum Symphony:

Complete the post-upgrade tasks. Some tasks, such as verifying and testing your IBM Spectrum Symphony upgrade, and upgrading your application to the latest version, you should complete. Other tasks, depend on whether you use certain features and options with IBM Spectrum Symphony.
Note: If you upgraded your cluster but have not yet upgraded your application to the latest version of IBM Spectrum Symphony, you can submit workload from your previous IBM Spectrum Symphony client.
If your cluster includes other products (for example, a multiple product environment can have IBM Spectrum Symphony, IBM Spectrum Conductor, and IBM Spectrum LSF® Standard Edition for Symphony, you can also upgrade any one of the products. For details, see Installing or upgrading IBM Spectrum Symphony with other IBM Spectrum Computing family products.