Selenium is an open source project with no formal guidance, making problem investigation time-intensive for testers. This article provides best practices for common issues during testing based on Selenium 1.0, which was released in June 2009.

Share:

Peng Cheng Yu (pengcy@cn.ibm.com), Software Testing Engineer, IBM

Peng YuPeng Cheng Yu is a software engineer in IBM China Development Lab. Working as a test lead of several globalization projects, he has experience in agile testing, quality assurance and test automation.



Zhang Chao Chao (chaochao@cn.ibm.com), Software Engineer, IBM

Chao Chao ZhangZhang Chao Chao is a software testing engineer in IBM China Development Lab. She has experience in agile testing and test automation.



07 December 2010

Also available in Japanese Portuguese

Introduction

Selenium is a robust set of tools that supports agile development of test automation for web-based applications across many platforms. It is an open source and lightweight automation tool that can be easily integrated into various projects, supporting multiple programming languages such the as .NET, Perl, Python, Ruby, and the Java™ programming language.


Testing an Ajax application with Selenium

Asynchronous JavaScript and XML (Ajax), is a web development technique for creating interactive web applications. A characteristic of an Ajax application is that it will not cause an entire page to reload each time. Instead, the browser will have an asynchronous call to the server to obtain the data and refresh only specific parts of the current page. To increase the web page's interactivity, response speed, and usability, some changes are required in testing Ajax applications.

We first act on the web page, then wait until the asynchronous call completes. After completion, verification can continue. At this point, the question of a suitable waiting time arises.

One choice is to simply pause for a fixed time in the test application, which may work in most cases. Under some circumstances, such as slow network throughput, the Ajax call is not completed after pausing for a specific time and will lead to the failure of the test cases. On the other hand, if the pause time is too long, it will make the test unacceptably slow.

Selenium has provided more-efficient ways to handle waiting. One possibility is to use the class com.thoughtworks.selenium.Wait to wait for an element or text to present or disappear from the page. The break condition can be defined for the wait in the until() function, or extend the Wait class to implement. Listing 1 is the sample code to use the Wait class. It will stop waiting if the condition is fulfilled, or return a timeout exception if it exceeds the maximum waiting time.

Listing 1. Wait for the element or text present
  Wait wait = new Wait() {
	public boolean until() {
	return selenium.isElementPresent(locator);
	// or selenium.isTextPresent(pattern);
	}
  };
  wait.wait("", timeoutInMilliseconds);

Another choice is to use the waitForCondition function in Selenium, to which a JavaScript snippet will be passed as a parameter. Once Selenium detects that the condition returns true, it will stop waiting. You can wait for some element or text to be present or not. JavaScript can be run in the application window get by the Selenium.browser.getCurrentWindow()function. Listing 2 is the sample code to check the state of the window. It only works in Firefox.

Listing 2. Wait the state of the window to be ready
String script = "var my_window = selenium.browserbot.getCurrentWindow();" 
script += "var bool;";
  script += "var readyState = (my_window.document.readyState);";
  script += "if (readyState == 'complete'){";
  script += "bool = 'true';";
  script += "}";
  script += "bool;";
  selenium.waitForCondition(script, timeoutInMilliseconds);

How to support Dojo applications

Dojo is a commonly used JavaScript tool kit to construct dynamic web interfaces. A key point in using Selenium to test Dojo applications is recognizing Dojo widgets and recording their actions. Author-defined Dojo widgets are in the abstract level. When the page is running, it will convert Dojo widgets to basic HTML code. There are many HTML codes auto-generated by Dojo, thus the recognition of Dojo widgets may have some differences with traditional HTML widgets.

Actions performed on Dojo widgets, including text field, button, checkbox, and radio button, may behave the same as with HTML widgets. On the combo box, however, date selector and other additional widgets provided by Dojo may require specific handling.

Figure 1. A Dojo combo box
Screenshot shows a combo box with several choices.

Use the Selenium IDE to record the selected action on the combo box provided in Figure 1. Click the down arrow and a drop-down list will appear. Select the third item Stack(SWG). The recorded scripts are provided in Figure 2.

Figure 2. Scripts recorded by Selenium IDE
Screenshot shows information that has been recorded by Selenium, such as mouse clicks

Sometimes, only the second line of the scripts will be generated by the IDE. In that case, add the action of clicking the arrow button. For the above scripts, if the first line is replayed, it should generate the drop-down list. However, it performs nothing. Click does not actually perform the click action for multiple Dojo widgets. Change the click(locator) to clickAt(locator, coordString), or MouseDown(locator) and MouseUp(locator).

Wait time should be added for the drop-down list. As the scripts show in Figure 2, the click action to the selected item will be performed right after clicking the down-arrow button. It will probably fail because the drop-down list has not appeared. Simply add a pause command, or use the waitFor command to wait for the item element to appear and continue on the next command.

Modified scripts that will automate selection on Dojo combo box are shown in Figure 3.

Figure 3. IDE scripts to select in Dojo combo box after modification
Screenshot shows that the clickAt, waitForElement and ClickAt commands are modified

Codes in RC are shown in Listing 3.

Listing 3. RC codes to automate selection on Dojo combo box
  selenium.clickAt("//div[@id='widget_offeringType']/div/div",””);
  selenium.waitForCondition("selenium.isElementPresent(\"offeringType_popup2\")", "2000");
  selenium.clickAt("offeringType_popup2",””);
Figure 4. A date selector
Screenshot shows a conventional month-calendar view for a date selector

For the example of date selector in Figure 4, the actions performed may not be recorded by the IDE. Write the RC codes as follows in Listing 4.

Listing 4. RC codes to automate selection
//click on the date field by id you defined;
selenium.clickAt("dateBox",""); 
//wait for the drop down date box by id;
selenium.waitForCondition("selenium.isElementPresent(\"widget_dateBox_dropdown\")", \
"2000"); 
//click previous year 2008;
selenium.clickAt("//span[contains(@class,'dijitCalendarPreviousYear')]", "");
//click on the month increase; 
//previous month would contains  ‘dijitCalendarIncrease’.
selenium.clickAt("//img[contains(@class,'dijitCalendarIncrease')]","");
//click on the date such as 28 of current month; If you do not specify 
//the td with the attribute of current month class, it will click \
on the //first 28 of previous month;
selenium.click("//td[contains(@class,'dijitCalendarCurrentMonth')]/span[text()='28']");

As the example shows, Dojo applications cannot be tested by simple IDE recording. These scripts are probably not able to pass. There could be some missing actions, or the actions do not really work. The scripts should be adjusted to make them run smoothly in IDE and RC. For complex Dojo widgets, a possible solution is to use the runScript(String) function, as Selenium provides good support for JavaScript. Listing 5 provides a JavaScript statement to simulate combo box selection.

Listing 5. Run JavaScript statements to select on combo box
selenium.runScript("dijit.byId(\"offeringType\").setValue(\"Stack(SWG)");");

How to build Selenium test with Ant

Integration tools, such as Ant, are convenient to build Selenium tests and run test cases smoothly without starting up the Selenium server stand-alone. If the Selenium test is driven by TestNG, define the TestNG Ant task as shown in Listing 6. In Listing 6, classpath supposes to be the file path of the TestNG.jar file.

Listing 6. TestNG Ant task
<taskdef resource="testngtasks" classpath="testng.jar"/>       

The main target is to start the server, run the tests, then stop the server. These tasks are implemented in sequence as defined in bulid.xml in Listing 7.

Listing 7. Ant task to start server, run test cases and stop server
<target name="run_test" description="start,run and stop" depends="dist">
<parallel>
 <antcall target="start-server" /> 
<sequential>
 <echo taskname="waitfor" message="Waitforproxy server launch" /> 
 <waitfor maxwait="2" maxwaitunit="minute" checkevery="100">
 <http url="http://localhost:4444/selenium-server/driver/?cmd=testComplete" /> 
 </waitfor>
 <antcall target="runTestNG" /> 
 <antcall target="stop-server" /> 
 </sequential>
 </parallel>
 </target>

It is preferable to use the waitfor task to test whether the Selenium server has successfully started, rather than pausing for a fixed time. If the URL http://localhost:4444/selenium-server/driver/?cmd=testComplete is available, it means that Selenium has launched successfully. In Listing 7, it waits up to two minutes and checks every 100 milliseconds for the Selenium server on the local host to serve up the complete URL.

The details for the start-server task are defined in Listing 8. Firefox profile template location and other arguments can be specified in the tag <arg>.

Listing 8. Ant task to start server in detail
<target name="start-server">
<java jar="lib/selenium-server.jar" fork="true">
 <arg line="-firefoxProfileTemplate ${selenium}/profile/" /> 
 </java>
 </target>

The details of the runTestNG task are defined in Listing 9. The commonly used attributes for the testng task include outputDir and xmlfileset. The attribute outputDir is used to set the output report location. The attribute xmlfileset is used to include startup XML files. Refer to the TestNG formal site for more options.

Listing 9. Ant task to run test cases
<target name="runTestNG">
<testng outputDir="${testng.report.dir}" sourcedir="${build}" 
classpathref="run.cp" haltOnfailure="true">
 <xmlfileset dir="${build}" includes="testng.xml" /> 
 </testng>
 </target>

Details of the stop-server task are defined in Listing 10.

Listing 10. Ant task to stop Selenium server
<target name="stop-server">
 <get taskname="selenium-shutdown" 
src="http://localhost:4444/selenium-server/driver/?cmd=shutDown" ignoreerrors="true" /> 
 <echo taskname="selenium-shutdown" message=" Errors during shutdown are expected" /> 
 </target>

The key tasks are listed above. Combine them to your build file to make well-integrated tests with Ant.


How to support testing HTTPS websites

As the Internet increasingly emphasizes information security, more web applications are using SSL authentication. Selenium IDE supports HTTPS by default, but Selenium RC does not. The solution varies in Internet Explorer and Firefox.

For IE, install a certificate in the SSL support folder under the setup directory. Use *iehta run mode if the version in use is prior to Selenium-RC 1.0 beta 2, and *iexplore run mode for Selenium-RC 1.0 beta 2 or later.

If a security warning appears when testing an HTTPS website as shown below, click View Certificate and install the certificate of the HTTPS website. If the warning continues to pop up, consider configuring in IE. Open Tool > Internet Options > Advanced, and uncheck Warn about invalid site certificates and Check for publisher's certificate revocation under the security category.

Figure 5. Security warning when testing HTTPS websites
Screenshot shows the Security Alert advising that the certificate is not verified

Creating a new Firefox profile

For Firefox, follow the steps below to create a customized profile, then restart the server:

  1. Close down any running Firefox instances.
  2. Start Firefox with the profile manager: firefox -ProfileManager.
  3. Create a new profile. When prompted, choose a directory for the profile. Place it inside the project folder.
  4. Select the profile and run Firefox.
  5. Navigate to the HTTPS URL with the self-signed certificate you will test against. Accept the certificate when prompted. This will create an exception in the profile.
  6. Close the browser.
  7. Go to the Firefox profile directory.
  8. Delete everything in the directory except for the cert_override.txt and cert8.db files.

By default, Selenium will create a new profile when launching the instance of Firefox. When you start the server with the parameter -firefoxProfileTemplate /path/to/profile/dir, Selenium will use a partial profile (with certificate exceptions) as a basis for minting the new profile. This will provide the certificate exceptions without additional clutter from using an entire profile. Pay attention to launching Firefox with the *firefox mode in Selenium RC 1.0 Beta 2 or later, and *chrome mode in versions prior to Selenium RC 1.0 Beta 2.

For the run modes, *chrome or *iehta were the experimental modes that supported HTTPS and the handling of security pop-ups in earlier versions of Selenium RC. They were stabilized to *firefox and *iexplore run modes since Selenium-RC 1.0 beta 2. Be careful to use the run mode according to the Selenium version in use.


How to effectively recognize web elements without ID property

Using a meaningful ID or name is an efficient and convenient way to locate elements. It can also improve the readability of test cases. However, for each element to have a meaningful and unique ID, especially for the dynamic element. Selenium has provided multiple strategies to recognize the elements, such as Xpath, DOM, and CSS.

Following is a sample using three strategies to locate an element in a dynamic table provided in Figure 6. The HTML codes are in Listing 11.

Figure 6. Dynamic table sample
Screenshot shows a simple table created dynamically with two columns and edit links
Listing 11. HTML code for the first table column
	<table id="test_table" border="1">
		<tbody>
        <tr>
            <td align="left">
                <div class="test_class">Test 1</div>
            </td>
            <td align="center" style="vertical-align: top;">
                <table id="AUTOGENBOOKMARK_4">
                    <tbody>
                        <tr>
                            <td align="center" style="vertical-align: top;">
                                <div>
                                  <img alt="supported" src="supported.png"/>
                                </div>
                            </td>
                            <td align="center" style="vertical-align: top;">
                                <div>
                                   <a href="test?name=test1">edit</a>
                                </div>
                            </td>
	…….

Xpath is a simple way to find elements with no specific ID or name.

  • If an attribute other than ID or name is known, use @attribute=value to locate the elements directly.
  • If only certain parts of attribute value are known, use contains(attribute, value) to locate the elements.
  • If the elements have no specified attributes, search for the nearest parent elements that have specified attributes with Firebug, then use Xpath to begin from this element to locate the desired element.
Table 1. Xpath expression to locate elements
Locate element Xpath expression
First column of row n//table[@id='test_table']//tr[n]/td
The image of row n//table[@id='test_table']//tr[n]//img
The edit link of ‘Test 1’ //a[contains(@href,test1)]

Table 1 shows the Xpath expressions of locating elements. With the help of Firebug, Xpath can locate elements and duplicated elements. Selenium IDE will adopt Xpath when the elements have no IDs and names. While Xpath assists in consistency with recorded scripts, it has a high dependence on the structure of the web page. This makes test cases unreadable and increases maintenance. Additionally, it might be too slow to run test cases that have multiple, complicated Xpath — expressions in Internet Explorer 7 and Internet Explorer 8. In such instances, change Xpath to DOM, which is another efficient locating strategy.

DOM is short for Document Object Model. Selenium allows you to traverse the HTML DOM with JavaScript. Java's flexibility allows for multiple statements in DOM expressions, separated by semicolons, and define functions in statements.

Table 2. DOM expression to locate elements
Locate element DOM expression
First column of row n dom=document.getElementById('test_table').rows[n-1].cells[0]
The image of row n dom=element=document.getElementById('test_table').rows[n-1].cells[1]; element.getElementsByTagName('IMG')[0]
The edit link of ‘Test 1’
dom=function test(){
var array=document.getElementsByTagName('a');
var element;for(var i=0; i<array.length; i++)
{if(array[i].attributes.getNamedItem("href").\
value.indexOf('test2')!=-1){element=array[i];break;}}return element}; test()

Table 2 shows the DOM expressions of locating elements. The DOM locator also has good performance in Firefox and Internet Explorer. Some JavaScript knowledge is required to organize DOM expressions. Sometimes the DOM expressions are too long for complicated elements and are hard to read (see the expression for the edit link of Test 1 mentioned in Table 2).

CSS locator is used to select elements with CSS selectors. CSS locator is efficiently utilized when HTML codes have good style. Sample expressions are shown in Table 3.

Table 3. CSS expression to locate elements
Locate element CSS expression
First column of row ncss=#test_table .test_class:nth-child(n)
The image of row n
css=#test_table  tr:nth-child(n) > td:nth-child(2) > 
table td:nth-child(1) > div >  img
The edit link of ‘Test 1’ css=a[href*='test2']

In general, choose familiar locator expressions and be consistent in script structure. If there are multiple expressions to execute, use the most effective way to locate the elements in web pages.


How to handle a pop-up window

In general, actions are performed in the main window launched by Selenium. If you want to perform actions in a new window generated by the window.open function, change the focus to the new window. After performing actions in a pop-up window, focus back to the main window. The procedure to handle a pop-up window is defined in Listing 12.

Listing 12. Sample code to handle a pop-up window
 //wait for the popup window with timeout;
selenium.waitForPopUp(windowname, timeout); 
//select the pop up window
selenium.selectWindow(popupWindowIdentifier);
//perform action on popup window and close the window;
.... 
//return to the main window use 'null' 
selenium.selectWindow(null);

The windowname is the second parameter to invoke the window.open function. The popupwindowIdentifier mentioned above is a window identifier, which can be window ID, window name, title=the title of the window, or var=javascript variable. If the attribute of the pop-up window is unknown, but is actually defined, use getAllWindowIds(), getAllWindowNames(), or getAttributeFromAllWindows()functions to retrieve the attributes of the pop-up window.

In the latest version of Selenium RC 1.0.1, Selenium has added methods like selectPopUp(String) and deselectPopUp(), the function of which has been provided by selectWindow(String) in earlier versions.

Listing 13. Pop-up functions to handle a pop-up window
 //wait for the popup window with timeout;
selenium.waitForPopUp(“”, timeout); 
//same as selenium.selectWindow
selenium.selectPopUp(“”);
//perform action on popup window and close the window;
.... 
//same as selenium.selectWindow(null);
selenium.deselectPopUp();

Listing 13 shows the simplest way to handle a pop-up window. You can leave the first parameter empty in the waitForPopUp and selectPopUp functions. If multiple windows pop up simultaneously, specify the window attribute.


How to handle upload/download file window

Selenium uses JavaScript to simulate the actions. Thus, it does not support browser elements such as upload window, download window, or authentication window. For accidental windows, configure the browser to skip pop-up windows.

Figure 7. Security information window
Screenshot shows the Security Information popup advising that the page contains both secure and nonsecure items

The solution to skip the security information window in Figure 7 is to open Tools > Internet Options > Custom Level. Then enable the item Display mixed content.

Configuring Internet Explorer to escape the accidental window will reduce or eliminate unnecessary handling when running test cases. If Firefox is configured, however, save it as a new profile and start the server with the customized profile. The reason for that is mentioned in the section regarding testing HTTPS websites.

For the upload/download windows, it is preferable to deal with them rather than skip them. To avoid the limitations of Selenium, one suggestion is to use the Java robot AutoIt to handle file upload and download problems. AutoIt is designed to automate Window GUI operations. It can recognize most Window GUI, provide many APIs, and can easily convert to .exe files, which can run directly or be invoked in Java code. Listing 14 illustrates the scripts to handle file upload. The steps of scripts:

  1. Determine the upload window title according to the browser type.
  2. Activate the upload window.
  3. Put the file path to the edit box.
  4. Submit it.
Listing 14. AutoIt scripts to handle upload
;first make sure the number of arguments passed into the scripts is more than 1 
If $CmdLine[0]<2 Then Exit EndIf
 handleUpload($CmdLine[1],$CmdLine[2])

;define a function to handle upload
 Func handleupload($browser, $uploadfile)
	 Dim $title                          ;declare a variable
            ;specify the upload window title according to the browser
            If $browser="IE" Then                  ; stands for IE;
 	      $title="Select file"
            Else                                 ; stands for Firefox
	       $title="File upload"
            EndIf
 
            if WinWait($title,"",4) Then ;wait for window with 
title attribute for 4 seconds;
                   WinActivate($title)                  ;active the window;
                   ControlSetText($title,"","Edit1",$uploadfile)   ;put the 
file path into the textfield  
                   ControlClick($title,"","Button2")                ;click the OK 
or Save button
            Else
	        Return False
            EndIf
 EndFunc 

In the Java code, define a function to execute the .exe file written by AutoIt and invoke the function after clicking browse.

Listing 15. Execute .exe file written by AutoIt
public void handleUpload(String browser, String filepath) {
	String execute_file = "D:\\scripts\\upload.exe";
	String cmd = "\"" + execute_file + "\"" + " " + "\"" + browser + "\""
				+ " " + "\"" + filepath + "\""; //with arguments
	try {
		Process p = Runtime.getRuntime().exec(cmd);
		p.waitFor(); //wait for the upload.exe to complete
	} catch (Exception e) {
		e.printStackTrace();
	}
}

Listing 16 is the AutoIt script to handle a download window in Internet Explorer. Download scripts vary in Internet Explorer and Firefox.

Listing 16. AutoIt scripts to handle download in Internet Explorer
If $CmdLine[0]<1 Then Exit EndIf
handleDownload($CmdLine[1])
Func handleDownload($SaveAsFileName)
Dim $download_title="File Download" 
If WinWait($download_title,"",4) Then
    WinActivate($download_title)
    Sleep (1000)
    ControlClick($download_title,"","Button2","")
    Dim $save_title="Save As"
    WinWaitActive($save_title,"",4)
    ControlSetText($save_title,"","Edit1", $saveAsFileName)
    Sleep(1000)
    if FileExists ($SaveAsFileName) Then
	FileDelete($SaveAsFileName)
	EndIf
    ControlClick($save_title, "","Button2","")
    Return TestFileExists($SaveAsFileName)
Else
    Return False
EndIf
EndFunc

AutoIt scripts are easy to write, but are dependent on the browser type and version as window title and window control class vary in different browsers and editions.


How to verify alert/confirmation/prompt information

For the alert dialog generated by window.alert(), use selenium.getAlert() to retrieve the message of a JavaScript alert generated during the previous action. The function will fail if there is no alert. Getting an alert has the same effect as manually clicking OK.

For the confirmation dialog generated by window.confirmation(), use selenium.getConfirmation() to retrieve the message of a JavaScript confirmation dialog generated during the previous action. By default, the function will return true, having the same effect as manually clicking OK. This can be changed by a prior execution of the chooseCancelOnNextConfirmation command.

For the prompt dialog generated by window.prompt(), use selenium.getPromt() to retrieve the message of a JavaScript question prompt dialog generated during the previous action. Successful handling of the prompt requires prior execution of the answerOnNextPrompt command.

JavaScript alerts will not pop up as visible dialogs in Selenium. Failure to handle these pop-up dialogs will cause exceptions stating that there are unexpected alerts. This will fail test cases.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Open source on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Open source
ArticleID=593907
ArticleTitle=Automated web testing with Selenium
publish-date=12072010