Installing Caffe for Linux on Power Systems Servers

Use this task to install Caffe for Linux on Power Systems Servers systems. You must install the Caffe deep-learning framework and related packages. Caffe is used for model training and defect classification.

Procedure

  1. Install the packages that are required for Caffe by using the following commands:
    sudo yum clean all
    sudo yum update
    sudo yum install upgrade
    sudo yum install -y libboost-*
    sudo yum install -y gflags-devel glog-devel lmdb-devel
    sudo yum install -y python-pip
    sudo yum install -y python-devel
    sudo yum install -y opencv-devel
    sudo yum makecache
    sudo yum install -y protobuf-devel leveldb-devel lmdb-devel snappy-devel opencv-devel boost-devel hdf5-devel atlas-devel glog-devel gflags-devel
    sudo yum install libpng-devel
    sudo yum install freetype-devel
    sudo yum install libjpeg-turbo-devel
    sudo yum install opencv-python
    sudo rpm -e --nodeps numpy
    sudo pip install numpy
    pip install --upgrade pip
    sudo pip install flask_httpauth
    sudo pip install gevent
    sudo pip install pyinotify
    ln -s /usr/local/cuda-10.0 /usr/local/cuda
    pip install scikit-image
    sudo pip install tornado
  2. Link the Atlas library by using the following commands:
    ln -fs /usr/lib64/atlas/libsatlas.so /usr/lib64/libatlas.so
    ln -fs /usr/lib64/atlas/libsatlas.so /usr/lib64/libcblas.so
  3. Download the Caffe source code by using the following command:
    wget https://github.com/BVLC/caffe/archive/1.0.zip
  4. Unpack the package and enter the package directory by using the following commands:
    unzip 1.0.zip
    cd ./caffe-1.0
  5. Replace the following Caffe files:
    • In the /include/caffe/util/cudnn.hpp directory, replace the cudnn.hpp file with the newest cudnn.hpp file that is in the Caffe repository on GitHub: github.com/BVLC/caffe.git
    • Replace all of the cudnn files that are in the /src/caffe/layers folder with the newest cudnn files that are in the Caffe repository on GitHub: github.com/BVLC/caffe.git

    For example, run the following commands:
    cp -rf /root/source/caffe-git/caffe-master/include/caffe/util/cudnn.hpp /usr/local/caffe-1.0/include/caffe/util/
    cp -rf /root/source/caffe-git/caffe-master/src/caffe/layers/cudnn_* /usr/local/caffe-1.0/src/caffe/layers/
    cp -rf /root/source/caffe-git/caffe-master/include/caffe/layers/cudnn_* /usr/local/p/usr/local/caffe-1.0/include/caffe/layers/
  6. Make a copy of the make configuration file by using the following command:
    cp Makefile.config.example Makefile.config
  7. Add the following variables in the Makefile.config file:
    USE_CUDNN := 1
    CUDA_DIR := /usr/local/cuda
    PYTHON_INCLUDE := /usr/include/python2.7 \
         /usr/lib64/python2.7/site-packages/numpy/core/include/
    PYTHON_LIB := /usr/lib/gcc/ppc64le-redhat-linux/4.8.5/
    WITH_PYTHON_LAYER := 1
    INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include 
    /usr/local/cuda-10.0/targets/ppc64le-linux/include/
    LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib \
        /usr/lib64 /usr/local/lib64
    Change CUDA_ARCH to the following text:
    CUDA_ARCH := -gencode arch=compute_30,code=sm_30 
    -gencode arch=compute_35,code=sm_35 
    -gencode arch=compute_50,code=sm_50 
    -gencode arch=compute_52,code=sm_52 
    -gencode arch=compute_60,code=sm_60 
    -gencode arch=compute_61,code=sm_61 
    -gencode arch=compute_61,code=compute_61
  8. Install the required Python packages in the caffe-1.0/python directory by using the following commands:
    cd caffe-installation-path/caffe-1.0/python
    for req in $(cat requirements.txt); do sudo -H pip install $req --upgrade; done
    where caffe-installation-path is the Caffe deployment path.
  9. Open the Makefile in the caffe-installation-path directory and change the parameter NVCCFLAGS to the following setting:
    NVCCFLAGS += -D_FORCE_INLINES -ccbin=$(CXX) -Xcompiler -fPIC $(COMMON_FLAGS)
  10. In the main Caffe directory caffe-1.0, begin the Caffe build and installation by using the following commands:
    make all
    make test
    make runtest
    make pycaffe
    make distribute
  11. Add the following line to the ~/.bashrc script:
    export PYTHONPATH="/usr/lib/python2.7:caffe-installation-path/caffe-1.0/python:$PYTHONPATH"
    where caffe-installation-path is the Caffe deployment path.
  12. Run the following post-installation tests:
    1. make runtest | tee -a runtest.out
    2. grep -i OK runtest.out | wc –l
      Caffe test output must be 2101
    3. python -c "import caffe"
      to test the Pycaffe installation
    4. tail -n 2 runtest.out
      The contents of runtest.out should contain the following text:
      [==========] 2101 tests from 277 test cases ran. (291548 ms total)
      [ PASSED ] 2101 tests