Adding a compute host

Add a compute host to your IBM Spectrum Conductor Deep Learning Impact environment that runs deep learning workloads.

Before you begin

Before installing the IBM Spectrum Conductor with Spark and IBM Spectrum Conductor Deep Learning Impact packages on the compute host, you must ensure that the compute host has the prerequisite packages installed and that all the compute hosts have the same packages installed.

Depending on the package that you are installing, make sure that the prerequisites are met.

About this task

Manually install both the IBM Spectrum Conductor with Spark and IBM Spectrum Conductor Deep Learning Impact packages on the compute host, then add the compute host to the cluster.

Procedure

  1. Log in to the compute host as root.
  2. Copy the installation files to the compute host.
  3. Define the following environment variables that are required by IBM Spectrum Conductor Deep Learning Impact. These values must be the same across all compute hosts.
    Environment variable Description
    CLUSTERADMIN Mandatory. Set to any valid operating user account, which then owns all installation files. For example:
    export CLUSTERADMIN=egoadmin
    DLI_SHARED_FS Mandatory. Set to the shared file system directory used by IBM Spectrum Conductor Deep Learning Impact for deep learning user data. The default is /gpfs/dlfs1. For example:
    export DLI_SHARED_FS=/gpfs/dlfs1
    Note: Ensure that the cluster administrator has read, write, and execute permissions on this directory.
    DLI_CONDA_HOME Mandatory for Linux for POWER LE (64-bit) only. Set to the Anaconda directory that is used by PowerAI deep learning frameworks.
    EGOCOMPUTEHOST Mandatory. Sets the host as a compute host. For example:
    export EGOCOMPUTEHOST=Y
  4. Install IBM Spectrum Conductor with Spark on the compute host.
    See Installing on a compute host.
  5. Run the IBM Spectrum Conductor Deep Learning Impact installer on the compute host.
    Note: IBM Spectrum Conductor Deep Learning Impact must be installed to the same location as IBM Spectrum Conductor with Spark.
    If you want to install with default settings, enter one of the following commands:
    For Linux for POWER LE (64-bit): 
    ./dli-1.1.0.0_ppc64le.bin
    For Linux (64-bit): 
    ./dli-1.1.0.0_x86_64.bin

    If you want to install to a different location other than the default, enter one of the following commands:

    For Linux for POWER LE (64-bit): 
    ./dli-1.1.0.0_ppc64le.bin --prefix install_location --dbpath dbpath_location
    For Linux (64-bit): 
    ./dli-1.1.0.0_x86_64.bin --prefix install_location --dbpath dbpath_location
    where:
    • --prefix install_location specifies the absolute path to the installation directory. The --prefix parameter is optional. If you install without the --prefix option, IBM Spectrum Conductor Deep Learning Impact is installed in its default directory: /opt/ibm/spectrumcomputing. Ensure that the path is set to the same directory as IBM Spectrum Conductor with Spark.
    • --dbpath dbpath_location sets the RPM database to a directory different from the default /var/lib/rpm. The --dbpath parameter is optional.
    • --silent enables silent installation. The --silent parameter is optional.
  6. After installation is complete, source the environment.
    • If you are running by using BASH, run:
      source $EGO_TOP/profile.platform
    • If you are running by using CSH, run: 
      source $EGO_TOP/cshrc.platform
  7. Log in to EGO as cluster administrator. 
    egosh user logon -u username -x password
  8. Add the compute host to the IBM Spectrum Conductor with Spark cluster. See Adding a host to a cluster.

Results

IBM Spectrum Conductor Deep Learning Impact was successfully installed on the compute host was added to the IBM Spectrum Conductor with Spark cluster. The compute host is ready to run deep learning workloads.