DevOps

Running Selenium tests from the command line on a Delivery Pipeline

Share this post:

When creating a cloud application, you often want to have automated tests for your basic UI functionality, like Selenium based tests. You can run Selenium tests through Sauce Labs or you can run them locally.
However, when you run them in a Delivery Pipeline in a DevOps toolchain on IBM Bluemix, you might see errors because the server doesn’t have a way to display the browser window.

This example creates a toolchain with a pipeline job that installs a virtual display buffer that allows Firefox to display. Then the simple python example will demonstrate running Selenium tests in the Delivery Pipeline.

Running tests on a Delivery Pipeline

To run Selenium tests in Delivery Pipeline, there are 4 main steps:

  1. Install the browser and virtual display buffer.
  2. Install the Selenium drivers for your test language.
  3. Run the virtual display buffer.
  4. Run the tests.

Install the browser and virtual display buffer

Make sure your build environment includes both tools that are needed. In this example, I’m using Firefox.

  1. Install Firefox
    sudo apt-get -y --force-yes update
    sudo apt-get -y --force-yes install firefox
    firefox -v
    
  2. Install Xfvb and some supporting fonts
    sudo apt-get -y --force-yes install xvfb xfonts-100dpi \
      xfonts-75dpi xfonts-cyrillic xorg dbus-x11
    

Install the Selenium drivers for your test language

In this example, the python Selenium package uses the geckodriver to drive the Firefox browser. Both the python Selenium package and geckodriver need to be installed.

  1. Install the geckodriver
    wget https://github.com/mozilla/geckodriver/releases/download/v0.17.0/geckodriver-v0.17.0-linux64.tar.gz
    mkdir bin
    tar xvzf geckodriver-v0.17.0-linux64.tar.gz -C bin
    export PATH=$(pwd)/bin:$PATH
    
  2. Install the python package
    sudo pip install -U Selenium
    

Run the virtual display buffer

The virtual display buffer program allows Firefox to behave like it’s displaying on your screen. The :10 indicates
what screen number the virtual display runs on

Xvfb :10 -ac &
export DISPLAY=:10

Run the tests

Now you can run the tests using whatever Selenium bindings you have installed.

py.test --junitxml results.xml tests/test_python_search.py || :

Try it

You can check out an example of a toolchain that builds a python test file and runs the test by clicking
the Create Toolchain button.

Create toolchain

Troubleshooting

Scenario: None of the Selenium tests run, and you see either of these error messages:

WebDriverException: Message: The browser appears to have exited before we could
connect. If you specified a log_file in the FirefoxBinary constructor, check it
for details.
WebDriverException: Message: Can't load the profile. Profile Dir: %s If you
specified a log_file in the FirefoxBinary constructor, check it for details.

Explanation: Your versions of Firefox and the Python Selenium driver might not be compatible. This problem is common.

Possible solution: Load either an older version of Firefox or the newest version.

  • To load an older version, enter this command:
    sudo apt-get install firefox=28.0+build2-0ubuntu2
    
  • To use the newest version of Firefox, enter this command:
    sudo add-apt-repository -y ppa:mozillateam/firefox-stable
    sudo apt-get -y update 
    

If you need to troubleshoot other issues, install a virtual machine (VM) by using a tool such as Vagrant. The Delivery Pipeline runs on Ubuntu 14.04, so a VM with that version of Ubuntu is suitable for testing and can save you time, as you won’t need to run a build every time that you want to test.

More How-tos Stories

How do I monitor my IBM Cloud applications?

If you are a service owner or first responder, you ask yourself "What’s going on with my IBM Cloud application?", "Are my customers satisfied with the service they’re getting?", "Has performance changed recently?" and so on. The answer begins with your organization's plan to design, deliver, operate, and control the IT and cloud services that it offers. This first post of the series begins with monitoring your cloud-based applications.

Continue reading

Customize Bluemix with your Git servers

You can now interact with your own GitHub Enterprise and GitLab instances from Bluemix public! Both the GitHub and GitLab tiles feature a new server dropdown menu, giving you the freedom to work with code on GitHub, GitLab, or in your own company’s GitHub Enterprise or GitLab instances.

Continue reading

Microservices: 5 Principles of Management

The benefits of a microservices architecture come with a price. The service management solution must deal with the architecture’s inherent dynamics, dependencies, and complexities to ensure that the application is available and performing. Ignoring these considerations could result in the microservice-based applications might behave worse than a monolithic application that was built in the traditional fashion. The principles of managing microservices explained in this post will help you avoid these pitfalls.

Continue reading