Installing Caffe for Linux on Power Systems Servers

Use this task to install Caffe for Linux on Power Systems Servers systems. You must install the Caffe deep-learning framework and related packages. Caffe is used for model training and defect classification.

1. Install the packages that are required for Caffe by using the following commands:
   sudo yum clean all
   sudo yum update
   sudo yum install upgrade
   sudo yum install -y libboost-*
   sudo yum install -y gflags-devel glog-devel lmdb-devel
   sudo yum install -y python-pip
   sudo yum install -y python-devel
   sudo yum install -y opencv-devel
   sudo yum makecache
   sudo yum install -y protobuf-devel leveldb-devel lmdb-devel snappy-devel opencv-devel boost-devel hdf5-devel atlas-devel glog-devel gflags-devel
   sudo yum install libpng-devel
   sudo yum install freetype-devel
   sudo yum install libjpeg-turbo-devel
   sudo yum install opencv-python
   sudo rpm -e --nodeps numpy
   sudo pip install numpy
   pip install --upgrade pip
   sudo pip install flask_httpauth
   sudo pip install gevent
   sudo pip install pyinotify
   ln -s /usr/local/cuda-10.0 /usr/local/cuda
   pip install scikit-image
   sudo pip install tornado
2. Link the Atlas library by using the following commands:
   ln -fs /usr/lib64/atlas/libsatlas.so /usr/lib64/libatlas.so
   ln -fs /usr/lib64/atlas/libsatlas.so /usr/lib64/libcblas.so
3. Download the Caffe source code by using the following command:
   wget https://github.com/BVLC/caffe/archive/1.0.zip
4. Unpack the package and enter the package directory by using the following commands:
   unzip 1.0.zip
   cd ./caffe-1.0
5. Replace the following Caffe files:
   • In the /include/caffe/util/cudnn.hpp directory, replace the cudnn.hpp file with the newest cudnn.hpp file that is in the Caffe repository on GitHub: github.com/BVLC/caffe.git
   • Replace all of the cudnn files that are in the /src/caffe/layers folder with the newest cudnn files that are in the Caffe repository on GitHub: github.com/BVLC/caffe.git

   
   For example, run the following commands:
   cp -rf /root/source/caffe-git/caffe-master/include/caffe/util/cudnn.hpp /usr/local/caffe-1.0/include/caffe/util/
   cp -rf /root/source/caffe-git/caffe-master/src/caffe/layers/cudnn_* /usr/local/caffe-1.0/src/caffe/layers/
   cp -rf /root/source/caffe-git/caffe-master/include/caffe/layers/cudnn_* /usr/local/p/usr/local/caffe-1.0/include/caffe/layers/
6. Make a copy of the make configuration file by using the following command:
   cp Makefile.config.example Makefile.config
7. Add the following variables in the Makefile.config file:
   

   USE_CUDNN := 1
   CUDA_DIR := /usr/local/cuda
   PYTHON_INCLUDE := /usr/include/python2.7 \
        /usr/lib64/python2.7/site-packages/numpy/core/include/
   PYTHON_LIB := /usr/lib/gcc/ppc64le-redhat-linux/4.8.5/
   WITH_PYTHON_LAYER := 1
   INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include 
   /usr/local/cuda-10.0/targets/ppc64le-linux/include/
   LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib \
       /usr/lib64 /usr/local/lib64

   Change CUDA_ARCH to the following text:

   CUDA_ARCH := -gencode arch=compute_30,code=sm_30 
   -gencode arch=compute_35,code=sm_35 
   -gencode arch=compute_50,code=sm_50 
   -gencode arch=compute_52,code=sm_52 
   -gencode arch=compute_60,code=sm_60 
   -gencode arch=compute_61,code=sm_61 
   -gencode arch=compute_61,code=compute_61

8. Install the required Python packages in the caffe-1.0/python directory by using the following commands:
   cd caffe-installation-path/caffe-1.0/python
   for req in $(cat requirements.txt); do sudo -H pip install $req --upgrade; done
   where caffe-installation-path is the Caffe deployment path.
9. Open the Makefile in the caffe-installation-path directory and change the parameter NVCCFLAGS to the following setting:
   NVCCFLAGS += -D_FORCE_INLINES -ccbin=$(CXX) -Xcompiler -fPIC $(COMMON_FLAGS)
10. In the main Caffe directory caffe-1.0, begin the Caffe build and installation by using the following commands:
    make all
    make test
    make runtest
    make pycaffe
    make distribute
11. Add the following line to the ~/.bashrc script:
    export PYTHONPATH="/usr/lib/python2.7:caffe-installation-path/caffe-1.0/python:$PYTHONPATH"
    where caffe-installation-path is the Caffe deployment path.
12. Run the following post-installation tests:
    1. make runtest | tee -a runtest.out
    2. grep -i OK runtest.out | wc –l
       Caffe test output must be 2101
    3. python -c "import caffe"
       to test the Pycaffe installation
    4. tail -n 2 runtest.out
       The contents of runtest.out should contain the following text:
       [==========] 2101 tests from 277 test cases ran. (291548 ms total)
       [ PASSED ] 2101 tests