Use Selenium Grid to enhance testing of web applications

Run automated, parallel tests in multiple environments

Selenium is a suite of tools designed to automate web application testing across many platforms. With Selenium Grid, which is part of the suite, you can transparently distribute your tests on multiple machines. In this article, learn how to run your tests in parallel, cutting down the time needed for running in-browser test suites. With real test projects and code samples, learn to run tests in multiple environments in parallel and speed up the testing of your web applications.

Zhen Li (bjlizhen@cn.ibm.com), Software Engineer, IBM

Li Zhen photoLi Zhen is a software engineer at the IBM China Development Lab working on datapower management. Her current focus is on web application test tools and test automation.



Yong Hu Sun (sunyongh@cn.ibm.com), Software Engineer, IBM

Sun Young Hu photoSun Yong Hu is a software engineer at the IBM China Development Lab. He works on datapower management and is team leader for the WebSphere Application Management Center SVT team. His current focus is on agile system verification test and test automation.



07 June 2011

Also available in Chinese Japanese Spanish

Introduction

Selenium Remote Control (RC), part of a suite of tools from the Selenium project, is an automated test framework for web applications. Selenium RC supports many programming languages and almost all browsers. The most important advantage of Selenium RC is the method in which the browser is driven. Unlike other web UI automation tools, Selenium RC drives the browser by JavaScript, so tests will not be interrupted by mouse events. Testers can do other tasks while the tests run.

Selenium Grid extends Selenium RC by running tests on different servers in parallel. It reduces the time, and subsequently the cost, for testing in various browsers under multiple operation systems.

Selenium tool suites are open source projects with limited formal documentation and guidance, so the learning curve might be steep. In this article, learn about testing with Selenium Grid (Version 1.0.6), programming in Java, and running with the TestNG testing framework.

Selenium Grid

Selenium Grid has two components: Selenium Hub and Remote Control. The hub receives requests from tests (the same as Selenium RC), and then allocates remote controls that are registered to the hub to each test, as shown in Figure 1:

Figure 1. Relationship of Selenium Hub and Selenium RC in Selenium Grid
Diagram of boxes, representing the omponents, linked by two-way arrows; Test scripts link to the Selenium Hub, which links to two separate Selenium RCss, each linking to their own browsers, which link to web applications

Testers can run tests without code changes in Selenium Grid. The whole process is transparent; testers do not need to know or care about the actual infrastructure.

If you're familiar with Selenium RC, you'll be able to easily run tests with Selenium Grid. You can also run existing code in parallel on multiple machines, which saves test execution time and quickly yields feedback of test results. However, Selenium Grid itself does not provide a parallel running strategy. If you want to take advantage of Selenium Grid, you need to write your selenium tests in parallel mode.


Preparing to use Selenium Grid

To get started with Selenium Grid, you first need to download and install it (see Resources for a link). Selenium Grid is a folder containing library JAR files, example files, Ant's build file, and other files. To run tests in Selenium Grid, you will need:

  • Ant, Version 1.7 or later, which is used for compiling, assembling, testing, and running Java applications (described in the build files as target and extension points).
  • JDK 5+.
  • Basic knowledge of Selenium RC, since Selenium Grid is based on Selenium RC. (See Resources for information about Selenium RC). Testers should know how to write and run tests using Selenium RC.

The example in this article uses Java technology and TestNG as the programming language and test framework. TestNG is designed to simplify a broad range of testing needs and to extend the JUnit. (See Resources for more information about TestNG.)A simple TestNG configure file is shown in Listing 1.

Listing 1. Simple TestNG configure file
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite1"  parallel="methods" thread-count="2">
<test name="Testcase20" >
        <classes>
            <class name="com.test.workflow.device.testcase20"/>
        </classes>
  </test>
</suite>

The <suite> tag can represent one TestNG XML file containing one or more tests. The <test> tag allows a test to contain one or more TestNG classes. The <class> tag signifies that a TestNG class can contain one or more test methods. (See Resources for more detailed information about tags.) The test method is defined in Java files, as shown in Listing 2:

Listing 2. Test method defined in Java file
@Test
public void testMethod()
{
    ....
}

Testers use the Java programming language and testing framework to write test scripts, and run TestNG classes from the command line using Ant. Let's take a look at the whole process of running TestNG using Ant.

  1. Run ant(target name) [build file name] in command line.
  2. Ant reads build.xml file to get project parameters, import files, classpath definition, target definition, and so on.
  3. Ant complies the Java source code to TestNG classes.
  4. Ant focuseson the target to run and gets the target configuration.
  5. Read TestNG configure file to get parameters, test classes name, and so on.
  6. Run TestNG classes as configured in TestNG configure file.

Sample test cases

This article focuses on how to run tests in parallel using Selenium Grid integrated with TestNG and Ant, so we'll use a simple application based on a PHP page. The test case is designed to simply click the tabs on the page. Figure 2 shows the example application web page.

Figure 2. Part of application web page
Screen shot of application web page, which contains the following tabs: Test Case, View Time, Run Scripts, Long Run, LA, s-Cat, Jmeter

Listing 3 provides the example source code for the tabs.

Listing 3. Source code for tabs
<div class="Tabs" style="width: 100%;">
  <a class="Current">Test Case</a>
  <a>View Time</a>
  <a>Run Scripts</a>
  <a>Long Run</a>
  <a>LA</a>
  <a>s-CAT</a>
  <a>Jmeter</a>
</div>

The test machines were prepared as follows.

IPOS Type
9.181.138.75win2003
9.181.138.186win2003
9.181.128.184 Linux

Put the test folder, whose content is shown in Figure 3, in the selenium-grid-1.0.6 directory.

Figure 3. Test case folder content
Screen shot of test case folder content, which includes subfolders src and target, and files build.xml and testing.xml

The src directory contains the Java source code files. The target directory contains all class files, after compilation, and report files. The build.xml file, shown in Listing 4, is an Ant build file that defines the target configuration. It includes which TestNG configure file to call, project parameters, the lib file path, and so on. The testng.xml file is the TestNG configuration file for the example test case.

Listing 4. build.xml
<project name="Selenium Grid Test" default="run" basedir=".">
  <description>Selenium Grid Test</description>
  <property name="rootdir" value="${basedir}/.."/>
  <property file="${rootdir}/project.properties"/>
  <property name="name" value="Selenium Grid Demo"/>
  <property name="artifact" value="selenium-grid-demo"/>
  <property name="version" value="SNAPSHOT"/>
  <property name="selenium.version" value="SET ME"/>
  <property name="build.src" location="src"/>

  <import file="${rootdir}/lib/build/common-build.xml" />

  <path id="compile.classpath">
    <fileset dir="${rootdir}/vendor">
      <include name="selenium-java-client-driver-${selenium.version}.jar"/>
      <include name="testng-5.7-jdk15.jar"/>
      <include name="commons-logging-1.0.4.jar"/>
    </fileset>
    <pathelement location="${rootdir}/tools/target/classes"/>
    <pathelement location=
    "${rootdir}/tools/target/dist/lib/selenium-grid-tools-standalone-${version}.jar"/>
    <pathelement location="${rootdir}/lib/selenium-grid-tools-standalone-${version}.jar"/>
    <pathelement path="${java.class.path}/"/>
  </path>

  <path id="runtime.classpath">
    <path refid="compile.classpath"/>
    <pathelement path="${build.output}/"/>
  </path>

  <target name="run" depends="compile" description="test">
       <java classpathref="runtime.classpath"
        classname="org.testng.TestNG"
        failonerror="true">
      <sysproperty key="java.security.policy" file="${rootdir}/lib/testng.policy"/>
      <arg value="-d" />
      <arg value="${basedir}/target/reports" />
      <arg value="-suitename" />
      <arg value="suite1" />
      <arg value="testng.xml"/>
    </java>

  </target>

  <target name="build" depends="compile"/>
  <target name="dist" depends="build"/>
  <target name="coverage-analysis" />
  
</project>

Running the sample test cases

Now that the environment and test data are set up, you can run three different scenarios of parallel testing:

Run same cases in different browsers on a server

To run the same test cases in different browsers on a server, go to the Selenium Grid distribution directory:

ant launch-hub

In a new window on the same machine, or on another machine, enter either:

ant launch-remote-control
ant Denvironment=*iehta launch-remote-control

or

ant  -Dhost=9.181.138.186 DhubURL=http://9.181.138.75:4444 launch-remote-control
ant  -Dhost=9.181.138.186 DhubURL=http://9.181.138.75:4444 
       -Denvironment=*iehta launch-remote-control

In a browser, go back to the Hub's console at http://localhost:4444/console. Now you should have two remote controls listed as available, similar to Figure 5.

Figure 4. Selenium Grid Hub console
Screen shot of Selenium Grid Hub console, showing two available remote controls

Put the test class source code from Listing 5 in the src directory.

Listing 5. Test case Java source code
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;
import com.thoughtworks.selenium.DefaultSelenium;
import com.thoughtworks.selenium.Selenium;
import org.testng.annotations.Parameters;

public class GridTest2 {
	String url = "http://9.181.138.186/index.php";
	private Selenium selenium= null;

	@Parameters({ "bs" })
	@BeforeClass
	public void startBroswer(String broswer) 
	{
		selenium= new DefaultSelenium("localhost", 4444, broswer, url);
		selenium.start();
		selenium.open(url);
		selenium.windowMaximize();
	
	}
	@Test
	public void test() throws InterruptedException
	{
		selenium.click("xpath=//a[contains(text(),\'LA\')]");
		Thread.sleep(10000);
	}
	@AfterClass
	public void clear() {
		selenium.stop();
	}	
}

Change the content of the testng.xml, as shown in Listing 6:

Listing 6. TestNG configure file content
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="suite1" verbose="1"  annotations="JDK" parallel="tests" thread-count="2">   
  <test name="test1">
	<parameter name="bs"  value="*firefox"/>
    <classes>
      <class name="GridTest2" />
    </classes>
 </test>
 <test name="test2">
	<parameter name="bs"  value="*iehta"/>
    <classes>
      <class name="GridTest2" />
    </classes>
 </test>
</suite>

The bs parameter in the testng.xml file defines that test1 runs in the Firefox browser and test2 runs in Internet Explorer. The parallel attribute defines that the tests run in parallel. The thread-count attribute defines the count of threads to run. There are only two tests defined, so the two tests will run at the same time.

Open a window, and go to the test directory:

ant run

The test case will run automatically.

Different cases run in parallel on different servers in different browsers

To run different test cases in parallel on different servers on different browsers, go to the Selenium Grid distribution directory:

ant launch-hub

In a new window on the same machine, or in a window on another machine, enter:

ant launch-remote-control
ant Dport=6666 Denvironment=*iehta launch-remote-control

In a window on machine 9.181.138.186, enter:

ant  -Dhost=9.181.138.186 DhubURL=http://9.181.138.75:4444 launch-remote-control

In a window on machine 9.181.138.184, enter:

ant  -Dhost=9.181.138.184 DhubURL=http://9.181.138.75:4444 launch-remote-control

In a browser, go back to the Hub's console at http://localhost:4444/console. You should now have four remote controls listed as available. It should look similar to Figure 5.

Figure 5. Selenium Grid Hub console
Screen shot of Selenium Grid Hub console, showing four available remote controls

Put the test class source code in Listing 7 in the src directory.

Listing 7. Test case Java source code
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;
import com.thoughtworks.selenium.DefaultSelenium;
import com.thoughtworks.selenium.Selenium;

public class GridTest2(3/4) {
	String url = "http://9.181.138.186/index.php";
	private Selenium selenium= null;

	@Parameters({ "bs" })
	@BeforeClass
	public void startBroswer() 
	{
selenium= new DefaultSelenium("localhost", 4444, broswer, url);
		selenium.start();
		selenium.open(url);
		selenium.windowMaximize();
	
	}
	@Test
	public void test() throws InterruptedException
	{
		selenium.click("xpath=//a[contains(text(),\'View Time\')]");
		//any code is ok
		Thread.sleep(10000);
	}
	@AfterClass
	public void clear() {
		selenium.stop();
	}
}

Change the contents of the testng.xml file, as shown in Listing 8:

Listing 8. TestNG configure file content
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="suite1" verbose="1"  annotations="JDK" parallel="tests" thread-count="2" >   
  <test name="test2"  junit="false" >
    <parameter name="bs"  value="*iehta"/>
    <classes>
      <class name="GridTest2" />
    </classes>
 </test>
 <test name="test3"  junit="false">
    <parameter name="bs"  value="*firefox"/>
    <classes>
      <class name="GridTest3" />
    </classes>
 </test>
<test name="test4"  junit="false">
    <parameter name="bs"  value="*firefox"/>
    <classes>
      <class name="GridTest4" />
    </classes>
 </test>
</suite>

Only two test cases will run at the same time because there are only two threads defined. Though there are four remote controls, the hub will choose RC1 to run GridTest2 and any of two remaining remote controls to run GridTest3 and GridTest4. The parallel running of two tests is random. Figure 6 shows the running status of the hub.

Figure 6. Selenium Grid Hub console
Screen shot of Selenium Grid Hub console, showing two available remote controls and two active remote controls

If you want to run all three of these tests at one time, just change thread-count to 3.

Different methods run in parallel on different servers with same browser

To run different test cases in parallel on different servers in the same browser, go to the Selenium Grid distribution directory:

ant launch-hub

Open a window on machine 9.181.138.186, and enter:

ant  -Dhost=9.181.138.186 DhubURL=http://9.181.138.75:4444 launch-remote-control

In a window on machine 9.181.138.184, enter:

ant  -Dhost=9.181.138.184 DhubURL=http://9.181.138.75:4444 launch-remote-control

In a browser, go back to the Hub's console at http://localhost:4444/console. Now you should have three or four remote controls listed as available. Put the test class source code from Listing 9 in the src directory.

Listing 9. Test case Java source code
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;
import com.thoughtworks.selenium.DefaultSelenium;
import com.thoughtworks.selenium.Selenium;
import static com.thoughtworks.selenium.grid.tools.
    ThreadSafeSeleniumSessionStorage.closeSeleniumSession;
import static com.thoughtworks.selenium.grid.tools.
    ThreadSafeSeleniumSessionStorage.session;
import static com.thoughtworks.selenium.grid.tools.
    ThreadSafeSeleniumSessionStorage.startSeleniumSession;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Parameters;

public class GridTest {
	String url = "http://9.181.138.186/index.php";
	private Selenium selenium;  

    	@BeforeMethod(alwaysRun = true)
    	protected void startSession() {
        		startSeleniumSession("localhost", 4444, "*firefox", url);
        		selenium = session(); 
  		selenium.open("/");
   	  	selenium.windowMaximize();
    	}

   	 @AfterMethod(alwaysRun = true)
    	protected void closeSession() {
        		closeSeleniumSession();
    	}
	@Test
	public void test() throws InterruptedException
	{
		
		selenium.click("xpath=//a[contains(text(),\'View Time\')]");
		Thread.sleep(10000);
	}
	@Test
	public void test2() throws InterruptedException
	{
		selenium.click("xpath=//a[contains(text(),\'LA\')]");
		Thread.sleep(10000);
	}
	@Test
	public void test3() throws InterruptedException
	{
		selenium.click("xpath=//a[contains(text(),\'Test Case\')]");
		Thread.sleep(10000);
	}	
}

Change the contents of the testng.xml file, as shown in Listing 10.

This source code is very different from the common selenium RC scripts. It initializes a Selenium object as a session object, since the methods should be run separately, so Selenium should start the session before the method and close the session after the method.

Listing 10. TestNG configure file content
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="suite1" verbose="1"  parallel="methods" thread-count="2">   
  <test name="test"  junit="false">
    <classes>
      <class name="GridTest" />
    </classes>
 </test>
</suite>

The parallel attribute is changed to methods, which means that the methods in the class will run in parallel. The classes will run in sequence.


Viewing the test results

It's very easy to view the test results. Go to the target folder defined in the build.xml file, and then to the reports folder. Open index.html, and click on the suite name link. You should see the results, as shown in Figure 7 which is the result of the third test sample.

Figure 7. Test results
Screen shot of test results, which includes one table containing the number of tests passed/failed/skipped, date test(s) started on, total time, included groups, and excluded groups, and a second table detailing test method and time (in seconds) of passed tests

For more detail, click on the chronological link on the left. You can get detailed information, such as that shown in Figure 8:

Figure 8. Detailed test results
Screen shot of table containing detailed test results: time, delta, suite configuration, test configuration, class configuration, groups configuration, method configuration, test method, thread, and instances

Conclusion

In this article, you have learned how to use Selenium Grid to run Selenium tests in parallel. Doing your automated web application testing with Selenium Grid can save a lot of time and make thrifty use of resources. For example, as described in the second scenario, if a tester has 100 test cases and configures 10 threads to run test cases, it will save 10 times the effort and time. Of course, the number of remote controls is limited.

Parallel testing can also make regression testing more convenient. Hopefully you can take what you've learned in this article and efficiently speed up the testing of your web applications.

Resources

Learn

  • SeleniumHQ: Learn all about the Selenium suite of tools to automate web app testing across many platforms.
  • Selenium and Selenium RC documentation: This users guide introduces Selenium, teaches its features, and presents commonly used best practices accumulated from the Selenium community.
  • Selenium Grid: Read more about how the Selenium Grid can distribute your tests on multiple machines so you can run tests in parallel.
  • TestNG: Learn more about TestNG and how to define the TestNG Ant task.
  • Apache Ant: Read all about the Apache Ant project.
  • IBM technical events and webcasts: Stay current with developerWorks' technical events and webcasts.

Get products and technologies

  • Selenium Grid: Download Selenium Grid.
  • TestNG: Get TestNG, a flexible testing framework that can also be used to drive Selenium's tests.
  • Apache Ant: Get Apache Ant, a Java library and command-line tool to drive processes described in build files as targets and extension points dependent upon each other.

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Web development on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Web development
ArticleID=678200
ArticleTitle=Use Selenium Grid to enhance testing of web applications
publish-date=06072011