Installing IBM Spectrum LSF Suite for HPC on IBM Power and x86 64-bit hosts

Install an IBM Spectrum LSF Suite for HPC cluster in a mixed IBM Power (LSF server hosts) and x86 64-bit (LSF management hosts) environment.

About this task

Install IBM Spectrum LSF Suite for HPC with a primary LSF host, and hosts that act as secondary failover candidate, GUI server hosts, LSF Web Services hosts (starting in Fix Pack 15), and database host. These hosts must run IBM Power Linux or 64-bit x86 Linux.

The LSF suite installation repository is on another host. A high-availability shared directory is configured on a separate host. Several compute hosts (LSF_Servers role) are also installed running on both IBM Power and 64-bit x86.

This configuration provides failover of the LSF host. Jobs continue to run even if the primary LSF host becomes unavailable. This task shows you how to verify that the high-availability shared directory (HA_shared_dir) is available, how to set up host roles on each server, and how to verify your installation.

This installation sets up the deployer host so that it can deploy both IBM Power and 64-bit x86 hosts. This task also shows you how to set up host roles on each server, how to verify which operating system versions are running in your cluster, how to check the host architecture, make sure your cluster is ready to accept work.

1. Log in as root to the host you want to use as the deployer host, and make sure that you downloaded both the 64-bit x86 and package from IBM Passport Advantage.

   64-bit x86 Linux systems

   • lsfshpc10.2.0.0-x86_64.bin

   IBM Power Linux (Little Endian) systems

   • lsfshpc.0.0-ppc64le.bin

2. Run the .bin files to create the repositories for both machine architectures to enable installation of IBM Power systems from the deployer host.
3. Change to the /opt/ibm/lsf_installer/playbook directory to customize the installation.
   1. Edit the lsf-inventory file to set up host roles for your cluster.

      LSF_Masters, GUI_Hosts, LSF_WebService, and DB_Host

      Specify the names of primary and secondary LSF hosts in the LSF_Masters option. Specify the name of the secondary LSF candidate host in the GUI_Hosts, LSF_WebService, and DB_Host options. For example,

      [LSF_Masters]
      hosta
      hostb
      ...
      [GUI_Hosts]
      hostb
      ...
      [LSF_WebService]
      hostb
      ...
      [DB_Host]
      hostb

      LSF_Servers and LSF_Clients

      List your LSF server hosts in the LSF_Servers option, one host or host name expression per line. The expression in the following example configures two server hosts (hostc, which is an x86 host, and hostd, which is an IBM Power host):

      [LSF_Servers]
      hostc
      hostd

      LSF_Clients

      List some LSF client hosts for users to submit jobs from in the LSF_Clients option, one host or host name expression per line. The expression in the following example configures three client hosts (hostb1, hostb2, hostb3):

      [LSF_Clients]
      hostb[1:3]

   2. Edit the lsf-config.yml file to set the cluster name and other properties.
      For example,

      LSF
        # Set my_cluster_name to the name of the cluster.
        my_cluster_name: cluster1
      ...

      Remember: Make sure to maintain the indentation in the lsf-config.yml file.

      Note: In version 10.2.0.9 and later, to install Kibana for use with Elasticsearch, set the parameter Install_Kibana: True. Kibana will be installed on the first GUI_Host defined in lsf-inventory.
      Because the LSF GUI host runs on a separate host from the primary LSF host, you need to specify a shared directory (HA_shared_dir). The directory must be mounted on both hosts. For example,

      LSF
      ...
      #  HA_shared_dir: none
        HA_shared_dir: /shared/lsfsuite
      ...

      Leave system monitoring enabled (Enable_Monitoring: True). The secondary host is used as the database host, so do not set the JDBC_string option.

4. After you set up the lsf-inventory and lsf-config.yml files, run two pre-installation tests.
   1. Check the configuration file, and correct any errors.

      ansible-playbook -i lsf-inventory lsf-config-test.yml

   2. Run the pre-deployment test.

      ansible-playbook -i lsf-inventory lsf-predeploy-test.yml

      This test runs on each host to check network connectivity and host name resolution, minimum disk space, and available memory. The test takes a few minutes to run.

   Correct any errors from the tests and run them again before you run the installation.
5. Run the installation with the lsf-deploy.yml playbook to deploy your mixed-architecture cluster.

   ansible-playbook -i lsf-inventory lsf-deploy.yml

   Note: By using the '--limit' option, new LSF_Servers can be installed and existing LSF_Servers can be reinstalled.

   ansible-playbook -i lsf-inventory --limit {Some Host or hosts} lsf-deploy.yml

   When the installation is finished, it gives you a URL for the IBM Spectrum LSF Suite for HPC portal and a URL for the LSF Web Services API endpoint.

   For example,

   http://hostb.company.com:8080
   https://hostb.company.com:8448

   Where hostb is the name of the GUI host that you configured in the lsf-inventory file.
6. Run some commands to verify the installation.
   1. Check the machine architectures and operating systems of the installed hosts.

      The following command runs on all the hosts listed in the lsf-inventory file, and uses the shell module to run the cat command to list the Linux release versions.

      ansible all -i lsf-inventory -m shell -a "/etc/redha-release"

      The following command runs on all the hosts listed in the lsf-inventory file, and uses the shell module to run the arch command to list the machine architectures of all the hosts in the cluster.

      ansible all -i lsf-inventory -m shell -a "arch"

      You should see both x86_64 and ppc64le hosts listed in the output.
   2. Log out of the deployer host, and log in to a host in the cluster.
   3. Run the lsid to see your cluster name and host name.

      lsid
      IBM Spectrum LSF 10.1.0.3, Sep 13 2017
      Suite Edition:  IBM Spectrum LSF Suite for HPC 10.2.0
      Copyright International Business Machines Corp, 1992-2017.
      US Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
      
      My cluster name is cluster1
      My name is hosta

   4. Run the lshosts command to see both LSF hosts (they are members of the management group indicated by the mg resource) The two LSF server hosts are listed, and the three client hosts are also listed.

      lshosts
      HOST_NAME      type    model    cpuf ncpus   maxmem  maxswp  server RESOURCES 
      hosta        X86_64  Intel_EM   60.0    20    63.8G   31.9G    Yes  (mg)
      hostb        X86_64  Intel_EM   60.0    20    63.8G   31.9G    Yes  (mg)
      hostd       LINUXPP    POWER9  250.0     8    14.9G    3.9G    Yes  ()
      hostc        X86_64  Opteron8   60.0     1     7.9G      1G    Yes  ()
      hostb1       X86_64    PC6000  116.1     4    31.9G    3.9G     No  ()
      hostb2       X86_64    PC6000  116.1     4    31.9G    3.9G     No  ()
      hostb3       X86_64  Opteron8   60.0     1     7.9G      1G     No  ()

   5. Run the bhosts command to check that the status of each host is ok, and the cluster is ready to accept work.

      bhosts
      HOST_NAME          STATUS       JL/U    MAX  NJOBS    RUN  SSUSP  USUSP    RSV
      hostc              ok              -      1     0       0      0      0      0
      hostd              ok              -      8     0       0      0      0      0
      hosta              ok              -     20     0       0      0      0      0
      hostb              ok              -     20     0       0      0      0      0

7. Log in to the GUI as the lsfadmin user.

   If the lsfadmin user was created by the installation and did not exist in your system, you might need to create a password for lsfadmin with the passwd command.

   1. Open your browser and enter the GUI portal URL from the installation.
   2. Log in with the lsfadmin user or any other user that you know exists.
   On the Resources > Dashboard page, you can see the six hosts that you deployed to.