Contents


An open source solution for dockerizing STAF and LTP on IBM Power Systems

Comments

Docker is now an integral component of Linux® distributions. Hence, it is important for all system vendors to validate kernel stability during an integration test. Docker helps in validating standard regression tests by running under different Linux distributions on the same host. This helps in significantly reducing the VM infrastructure and shortening the test cycle by running the tests in a fully automated test framework. It is also important to understand the behavior of standard test frame work and test suites, while running inside a container.

This article describes how to get the Software Testing Automation Framework (STAF) and Linux Test Project (LTP) test framework to run inside a Docker container. These Docker containers run on supported Linux distributions on IBM® Power Systems™. The STAF and LTP framework is a widely used regression test combination to validate the behavior of Linux kernel and glibc components on the Linux distribution.

Software Testing Automation Framework (STAF)

The Software Testing Automation Framework (STAF) is an open source framework containing a set of built-in services that are especially suited to construct an automation solution. Services are reusable components that provide all the capabilities in STAF. Each STAF service provides a specific set of functions such as logging and monitoring, file system operations, process invocation, and resource management.

STAF Execution Engine (STAX) is another integral component that is built upon STAF, XML, and Python to automate the workload distribution, execution, result analysis, and parallel processing of your test cases.

Linux Test Project (LTP)

Linux Test Project (LTP) is an open source automated kernel test suite that is designed to validate the reliability, robustness, and stability of Linux. It mainly focuses on functionality, regression, and stress testing for the Linux kernel and related features.

Docker

Docker is an open source project based on Linux containers. It uses Linux kernel features such as namespaces and control groups to create containers on top of an operating system.

Prerequisites

This section lists the prerequisites required for dockerizing STAF and LTP.

Operating system level requirements

To install Docker, you need a 64-bit version Linux kernel. The following are the supported Linux distributions:

  • Ubuntu - Trusty 14.04, Xenial 16.04, Yakkety 16.10, and later
  • SLES - SLES 12.x, and later
  • RHEL - RHEL 7, and later

Open source images

To create the LTP image and STAF image, the following are the requirements:

  • LTP test suite
  • Latest STAF/STAX source
  • Open Java
  • Sample ltp.xml file

How to build STAF and LTP Docker image

This section explains two methods to build a Docker image. The first method describes how to install, configure, and start the required STAF and LTP services manually inside the container and commit by using the docker commit command. This is the preferred method while configuring any new dockerized software for the first time.

The second method uses a Docker file for automatically configuring the steps explained in first part. This is the recommended method that is used in most of the Docker deployment projects. However, the first method is to resolve the dependency and other challenges when you perform the configuration for the first time.

Creating a Docker image by using Docker commit

After installing any of the Linux distributions mentioned earlier, perform the following steps to create a Docker image:

  1. Create the container by running the following command:
    docker run -i -t --privileged --name=ltptest --hostname=ltptest ppc64le/ubuntu /bin/bash
  2. Download and copy the LTP test cases to the container.
    Complete the following steps on the host to download and build the LTP test bucket and then copy the LTP test cases to the Docker container:
    1. Download the latest stable LTP test suite to the host from http://linux-test-project.github.io/. In this example, ltp-full-20160920 is downloaded.
    2. Extract the tar.gz or .bz2 ltp packages by running the following command. After you run the command, you can see more information about the installation process in the INSTALL file in ltp-full-20160920.
      tar jxvf ltp-full-20120104.tar.bz2 or tar zxvf ltp-full-20120104.tar.gz
    3. Change to the LTP directory by running the following command:
       cd ltp-full-20160920
    4. Check the configuration options by running the following command:
       ./configure --help
    5. Build the 64-bit LTP on the IBM Power Systems server by running the following commands:
      [ltp-full-20160920]# ./configure --prefix=`pwd` --with-open-posix-testsuite CC="gcc -   m64" 
       	[ltp-full-20160920]# make all
      	[ltp-full-20160920]# make install
    6. Copy LTP test cases to the container.
      Test case location in the host: /root/ltp-full-20160920
      cd /root/ltp-full-20160920
      docker cp . ltptest:/usr/local/ltp
  3. Download, configure, and start STAF inside the container by completing the following steps:
    1. In the https://sourceforge.net/projects/staf/files/ link, download the STAF and STAX files from the staf and stax directories respectively.
      For example, files that are required are STAF3425-setup-linux-ppc64le.bin and STAXV3517.
    2. Extract STAX by running the following command:
       tar -xvf STAXV3517.tar
    3. Copy STAF3425-setup-linux-ppc64le.bin and STAX from the host to the container by running the following commands:
      docker cp STAF3425-setup-linux-ppc64le.bin <container-id>:/root/
      docker cp stax <container-id>:/root/

      From step d onwards, you need to continue inside the Docker container.
       docker exec -it container_id /bin/bash
    4. Install STAF by running the following command:
      apt-get install -y unzip (pre-req to help STAF install) ./root/STAF3425-setup-linux-ppc64le.bin -i silent -DACCEPT_LICENSE=1
    5. Create a directory under /usr/local/staf/services by running the following command:
       mkdir -p /usr/local/staf/ services
    6. Copy the extracted stax to the above services directory by running the following command:
       cp -r ./stax /usr/local/staf/services/
    7. Add the following lines to the /usr/local/staf/bin/STAF.cfg file.
      SERVICE STAX LIBRARY JSTAF EXECUTE /usr/local/staf/services/stax/STAX.jar \
        PARMS "EVENTGENERATION Disabled LOGTCNUMSTARTS Disabled LOGTCELAPSEDTIME Disabled LOGTCSTARTSTOP Disabled NUMTHREADS 10" OPTION JVMName=STAFJVM1 OPTION J2=-Xmx1024m
      SERVICE EVENT LIBRARY JSTAF EXECUTE /usr/local/staf/services/stax/STAFEvent.jar
    8. Remove any older version of Java, if it exists.
    9. Install open source Java by running the following commands:
      apt-get install -y default-jre
      
      apt-get install -y openjdk-8-jre
    10. Add the following lines to the /etc/profile.d/staf.sh file. Also, add it to /root/.bashrc.
      base_java_path='/usr/lib/jvm/java-8-openjdk-ppc64el/jre/bin'
      java_path=''  
      in_string=`export | grep PATH`
      in_string1=`echo "$in_string" | grep "declare -x PATH="`
      string=`echo "$in_string1" | grep "/usr/local/staf/bin"`
      if [ ! $? = 0 ]; then
         PATH=/usr/local/staf/bin:$PATH
      fi
      string=`echo "$in_string1" | grep "/usr/local/staf/tools"`
      if [ ! $? = 0 ]; then
         PATH=/usr/local/staf/tools:$PATH
      fi
      string=`echo "$in_string1" | grep $java_path`
      if [ ! $? = 0 ]; then
         PATH=$java_path:$PATH
      fi
      in_string1=`echo "$in_string" | grep "declare -x CLASSPATH="`
      string=`echo "$in_string1" | grep "/usr/local/staf/lib/JSTAF.jar"`
      if [ ! $? = 0 ]; then
         CLASSPATH=/usr/local/staf/lib/JSTAF.jar:$CLASSPATH
      fi
      string=`echo "$in_string1" | grep "/usr/local/staf/services/STAX.jar"`
      if [ ! $? = 0 ]; then
         CLASSPATH=/usr/local/staf/services/STAX.jar:$CLASSPATH
      fi
      string=`echo "$in_string1" | grep "/usr/local/staf/services/STAFEvent.jar"`
      if [ ! $? = 0 ]; then
         CLASSPATH=/usr/local/staf/services/STAFEvent.jar:$CLASSPATH
      fi
      string=`echo "$in_string1" | grep "/usr/local/staf/services/STAXMon.jar"`
      if [ ! $? = 0 ]; then
         CLASSPATH=/usr/local/staf/services/STAXMon.jar:$CLASSPATH
      fi
      string=`echo "$in_string" | grep "declare -x LD_LIBRARY_PATH=" | grep "/usr/local/staf/lib"`
      if [ ! $? = 0 ]; then
         LD_LIBRARY_PATH=/usr/local/staf/lib:$LD_LIBRARY_PATH
      fi
      STAFCONVDIR=/usr/local/staf/codepage
      export PATH CLASSPATH LD_LIBRARY_PATH STAFCONVDIR
      setsid /usr/local/staf/bin/STAFProc >/tmp/STAFProc.out 2>&1 &
      set -o vi
    11. Exit from the container and log in again.
  4. Download and copy the sample XML file to the container's location: /usr/local/staf/xml/ltp.xml.

    You can download the ltp.xml file from the sourceforge.net website or from the github repository.

  5. Start the LTP tests inside the container by using STAF:
    STAF local STAX execute file /usr/local/staf/xml/ltp.xml JOBNAME ltpstaf
  6. Commit the Docker image from the host by running the following command:
     docker commit <container ID> <repo name>

Creating a Docker image using Dockerfile

Dockerfile completes the configuration that are required to set up the STAF framework. Dockerfile also completes the configuration that is required for the LTP testbucket to be run from the STAF framework.

When a container is launched by using the image that is built from the Docker file, LTP tests are started by using the STAF framework.

You can find the Dockerfile, other configuration files and details on how to create ltp_staf docker container in the github repository at: https://github.com/ibmsoe/Dockerfiles/tree/master/ltp_staf.

Perform the following steps to build the image:

  1. Clone the ltp_staf directory in the https://github.com/ibmsoe/Dockerfiles/tree/master/ltp_staf website by running the following command:
    gitclone https://github.com/ibmsoe/Dockerfiles/tree/master/ltp_staf
  2. Change the directory by running the cd /STAF_LTP command.
  3. Build the Docker image by running the docker build -t <image_name> command.

Monitoring LTP tests

You can monitor the tests that are running inside the container using the following native STAF commands.

docker exec -it <container_id> /bin/bash
STAF LOCAL STAX LIST JOBS
STAF LOCAL STAX LIST JOB <job ID> testcases

Note: Test case result is inside the Docker container at the /var/log/staf/ltp location.

Disclaimer

Complete LTP test suites are not tested or included as part of this project. This project is only to show the enablement part of Docker, LTP, and STAF on IBM Power Systems server.

The following are the LTP test cases included for this project.

Table 1. LTP test cases
Admin_tools Stress.part2 Io_floppy Nw_under_ns
Can Stress.part3 ipc Pipes
Cap_bounds Syscalls tracing Power_management_tests
Commands Syscalls-ipc Kernel_misc Tom_tools
Connectors Timers Ltp-aiodio.part1 makefile
Containers Fs_ext4 Ltp-aiodio.part2 Pty
Controllers Fs_perms_simple Ltp-aiodio.part3 quickhit
Cpuhotplug Fs_readonly Ltp-aiodio.part4 Rpc_tests
Dio fsx Ltp-aio-stress.part1 Sched
Dma_thread_diotest hugetlb Ltp-aio-stress.part2 Scsi_debug.part1
fcntl-locktests hyperthreading mm sercurebits
fliecapts ima modules smack
fs io nptl Lvm.part2
Fs_bind Io_cd numa
Stress.part1 math Tirpc_tests
ltplite Lvm.part1

The sample XML (ltp.xml) file includes syscalls, fs, fsx, dio, mm, commands, ipc, sched, pty, and math test cases. The XML file can also be modified to include the above listed test cases. Edit the RUNTEST variable in the ltp.xml file.

For example:

"RUNTEST" default="['lvm.part2','power_management_tests','power_management_tests_exclusive','quickhit','scsi_debug.part1','securebits','smack','stress.part2','stress.part3']"

Note: When there is a change in the LTP, STAF, STAX, and Java builds the directories will change accordingly. You need to add the relevant proxies or repos, if your system is in a private network.

Acknowledgement

We would like to thank Pradipta K Banerjee STSM, Power Cloud and Docker Architect, Linux Technology Center for his guidance and help to publish this article.

Related resources


Downloadable resources


Comments

Sign in or register to add and subscribe to comments.

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Linux
ArticleID=1045663
ArticleTitle=An open source solution for dockerizing STAF and LTP on IBM Power Systems
publish-date=05102017