Preparing to install IBM Spectrum Conductor Deep Learning Impact for POWER9 with IBM PowerAI 1.5.1

Preparing to install IBM Spectrum Conductor Deep Learning Impact on POWER9 with IBM PowerAI 1.5.1.

Note: IBM PowerAI 1.5.0 and PowerAI 1.5.1 packages are no longer available. In order to install IBM Spectrum Conductor Deep Learning Impact 1.1.0 on POWER8 or POWER9, you must get IBM PowerAI Enterprise 1.1. Contact the IBM account or services team for more information, or go to the IBM PowerAI Enterprise website to learn more.

Before you begin

Before preparing to install IBM Spectrum Conductor Deep Learning Impact foPreparing to install IBM Spectrum Conductor Deep Learning Impact on POWER9 with IBM PowerAI 1.5.1. To install on POWER9 with PowerAI 1.5.1, your IBM Spectrum Conductor Deep Learning Impact installation must include iFix which adds POWER9 with PowerAI 1.5.1 support to your environment.r POWER9, make sure that you do the following:
  1. Make sure that you meet the required requirements , see Requirements for IBM Spectrum Conductor Deep Learning Impact 1.1.0.
  2. Make sure that you apply the RHSA-2018:1374 security fix available at https://access.redhat.com/errata/RHSA-2018:1374.
  3. Make sure that you satisfy all the IBM PowerAI 1.5.1 requirements. For IBM PowerAI installation requirements, see the IBM PowerAI website.
  4. Make sure that you satisfy all the IBM Spectrum Conductor with Spark installation requirements, see IBM Spectrum Conductor with Spark requirements.
  5. Make sure that you satisfy all the IBM Spectrum Conductor Deep Learning Impact installation requirements. The IBM Spectrum Conductor Deep Learning Impact installation requirements are as follows:
    • Ensure that you have root access to all hosts running deep learning workloads.
    • If a firewall is enabled, the following default ports must be granted access on all management hosts: 9243, 9280, 5000, 5001, 27017, and 6379. If you change these ports after installation, make sure to update firewall rules accordingly.

About this task

Complete the following prerequisites before installing IBM Spectrum Conductor Deep Learning Impact for POWER9.

Procedure

  1. Mount a shared file system as required by IBM Spectrum Conductor Deep Learning Impact.  The file system will be used for user data, such as datasets, tuning data, validation results, training models and more. 
    The requirements for the shared file system are following:
    • Minimum of 2 GB of free disk space
    • Cluster administrator account (the account that was specified by the CLUSTERADMIN variable during IBM Spectrum Conductor with Spark installation) must have read and write permissions
    • setfacl must be supported on the nodes mounting the file system
    To verify that you mounted the shared file system correctly, assuming that cluster administrator account is egoadmin and the mount point is /dli_shared_fs:
    1. First, export the environment variables:
      export CLUSTERADMIN=egoadmin
export ADMINGROUP=egoadmin
export DLI_SHARED_FS=/dli_shared_fs
    2. Then, run the following code:
      chown -Rh $CLUSTERADMIN:$ADMINGROUP $DLI_SHARED_FS
chmod g+s $DLI_SHARED_FS
chmod 777 $DLI_SHARED_FS
setfacl -d -m u:$CLUSTERADMIN:rwx $DLI_SHARED_FS
setfacl -d -m g::r-x $DLI_SHARED_FS
setfacl -d -m o::--- $DLI_SHARED_FS
    Note: The shared file system must be mounted to a clean directory. If you are reinstalling IBM Spectrum Conductor Deep Learning Impact, make sure that the directory specified is empty.
  2. Install IBM PowerAI on all cluster nodes. For more information about IBM PowerAI installation, see the IBM PowerAI website.
    Note: IBM PowerAI is installed locally under the /opt/DL directory.
  3. Download and install Anaconda 5.1. 
    wget https://repo.continuum.io/archive/Anaconda2-5.1.0-Linux-ppc64le.sh
bash Anaconda2-5.1.0-Linux-ppc64le.sh -b -p /opt/anaconda2
  4. Export the Anaconda path. 
    export PATH=/opt/anaconda2/bin:$PATH
  5. Create an Anaconda environment named dli for the installation of IBM Spectrum Conductor Deep Learning Impact. 
    conda create --name dli --yes pip python=2.7
  6. Install TensorFlow dependencies. 
    /opt/DL/tensorflow/bin/install_dependencies -n dli
  7. Install TensorBoard dependencies. 
    /opt/DL/tensorboard/bin/install_dependencies -n dli
  8. Install OpenCV and create symbolic links. 
    yum install opencv-devel python-devel opencv-python boost-python snappy-devel
ln -s /usr/lib64/python2.7/site-packages/cv.py /opt/anaconda2/envs/dli/lib/python2.7/site-packages/cv.py
ln -s /usr/lib64/python2.7/site-packages/cv.pyc /opt/anaconda2/envs/dli/lib/python2.7/site-packages/cv.pyc
ln -s /usr/lib64/python2.7/site-packages/cv.pyo /opt/anaconda2/envs/dli/lib/python2.7/site-packages/cv.pyo
ln -s /usr/lib64/python2.7/site-packages/cv2.so /opt/anaconda2/envs/dli/lib/python2.7/site-packages/cv2.so
  9. Install Fabric dependencies. 
    yum install openblas-devel glog-devel gflags-devel hdf5-devel leveldb-devel libsodium-devel lmdb-devel
  10. Install protobuf. 
    yum install protobuf-java protobuf-static protobuf-devel protobuf-lite protobuf-lite-static protobuf-lite-devel protobuf-vim
yum install gcc-c++ gcc-gfortran freetype-devel libpng-devel libffi-devel openssl-devel
  11. Activate dli and install dependencies. 
    source activate dli
conda install wheel==0.31.0
pip install setuptools==39.0.1
conda install cython==0.25.2 h5py==2.7.0 ipython==5.3.0 python-leveldb=0.194 python-lmdb==0.92 matplotlib==2.0.2 networkx==1.11 nose==1.3.7 pandas==0.20.3 pillow==4.1.1 python-dateutil==2.6.1 pyyaml==3.12 requests==2.13.0 scipy==1.0.1 six==1.11.0 scikit-image==0.13.0 redis-py==2.10.5 chardet==3.0.4
pip install Keras==2.0.5 easydict==1.6 python-gflags==2.0
source deactivate
  12. Create an Anaconda environment named dlmao for IBM Spectrum Conductor Deep Learning Impact training insights. 
    conda create --name dlmao --yes pip python=2.7
  13. Activate dlmao and dependencies. 
    source activate dlmao
conda install wheel==0.31.0
pip install setuptools==39.0.1
conda install numpy==1.12.1
conda install flask==0.12.2 flask-cors==3.0.3 pyopenssl==17.0.0 sqlalchemy==1.1.13 scipy==1.0.1
pip install alembic==0.8.2 elasticsearch==5.2.0 Flask-Script==2.0.5 Flask-HTTPAuth==3.2.2 mongoengine==0.11.0 pathlib==1.0.1 python-heatclient==1.2.0 python-keystoneclient==3.1.0 
source deactivate
  14. Verify that your IBM PowerAI deep learning frameworks are installed correctly.
    To test framework installation, run the test script provided by each framework:
    source activate dli 
<framework>-test
  15. Install IBM Spectrum Conductor with Spark using the shared or local installation method. See Installing IBM Spectrum Conductor with Spark.
    Important:
    • IBM Spectrum Conductor Deep Learning Impact must be installed using the same installation method (either shared or local) as IBM Spectrum Conductor with Spark.
    • If installing IBM Spectrum Conductor Deep Learning Impact to an existing IBM Spectrum Conductor with Spark environment, rolling upgrade for IBM Spectrum Conductor with Spark is not supported.
    • After installing IBM Spectrum Conductor with Spark, make sure to apply the latest IBM Spectrum Conductor with Spark fixes from IBM Fix Central.
  16. Entitle IBM Spectrum Conductor with Spark. See Entitling IBM Spectrum Conductor with Spark.

Results

The prerequisite packages are installed on the Linux for POWER LE (64-bit) host.

What to do next

Install IBM Spectrum Conductor Deep Learning Impact on the host.