Use IBM Rational Functional Tester to automatically capture bitmap images of multiple tooltips

During testing, we sometimes need take screen captures with multiple tooltips related to the application under test (AUT). For example, during translation verification testing (TVT), lots of tooltips might be necessary. Most of time, we attach multiple tooltips in the same bitmap image file to save the TVT tester time and effort. However, we can't identify the properties of the tooltip directly by using the Test Object Inspector. Another problem is when we attach multiple tooltips in the same base bitmap file, where is the proper position? If we do not know, there will be truncation or overlap after attachment. Huai Ying Xia and Zhi Jun Wang describe a way to solve these problems by using IBM Rational Functional Tester to automatically take screen captures with multiple tooltips attached.

Huai Ying Xia (xiahy@cn.ibm.com), Globalization Test Specialist, IBM China

author photoHuai Ying Xia is a globalization software engineer for IBM Notes email software. She has been working for IBM since 2012 and focuses on automation testing in multilingual environments, using IBM Rational Function Tester.



Zhi Jun Wang (wzhijun@cn.ibm.com), Software Engineer, IBM

author photoZhi Jun Wang is a globalization test leader for IBM Notes software. He has worked for IBM since 2010 and focuses on automation testing in translated products.



04 June 2013

Reasons to automate taking multiple screen captures of tooltips

During translation verification testing (TVT) in a multilingual environment, many screen captures need to be taken. This erodes the quality testing engineers' enthusiasm, and it also requires a lot of manual effort to complete. For example, multiple tooltips need to be attached to the same bitmap image, as shown in Figure 1. It takes a lot of time to do the split-join work repeatedly for multilingual environments. For IBM Notes software, for example, there are 29 languages, so we must take 29 screen captures for each language during the TVT test period.

Therefore, it is necessary to automate such repetitive work. However, for reasons that we explain in this article, we can't identify the properties of the tooltip directly by using the Test Object Inspector in IBM® Rational® Functional Tester. Another problem is that when attach multiple tooltips on the same base bitmap, where is the proper position? If it is not correct, there will be truncated or overlapped attachments. This article describes the basic method to solve these problems.

Figure 1 shows the result: A screen capture with multiple tooltips attached automatically by Rational Functional Tester. (The application under test is a web-based application.)

Figure 1. Example of bitmap file contents with multiple tooltips attached
One bitmap with eight tooltips attached

How to inspect the tooltip

Think about the regular automation testing process. We must use the properties, such as the object's ID, UI string, class name, and so forth, to recognize this object. So it is a prerequisite to inspect the properties of the object. Then Rational Functional Tester can identify this object during testing by comparing the run time object's properties with the properties described in the scripts. When you need to inspect elements that are showing up only on a page, if you put the mouse cursor over some area, such as tooltip element, the problem is that the Test Object Inspector still needs to hover over the desired test object. Therefore, the cursor is working with the Test Object Inspector, so the tooltip event can't be triggered. Thus, as Figure 2 shows, the Test Object Inspector can't inspect a pop-up tooltip element

Figure 2. The cursor is working only with the Test Object Inspector
Test Object Inspector identifies tooltip

When you use the Test Object Inspector to inspect the properties of an object, it must hover over the specified test object. However, the tooltip is hidden when you use it, so the Test Object Inspector can't inspect the pop-up tooltip's properties.

Usually, you can inspect the properties of the base hover object (over which the mouse hover, the tooltip will pop up) by the Test Object Inspector. The tooltip is close to its base hover object, so you can estimate the tooltip's coordinate according to its base hover object's properties. In most cases, the pop-up tooltip has the same fixed height, which you can also estimate according to its base hover object's properties. To get the properties of a pop-up tooltip, such as class name, first print all top windows' class names, height, and coordinates. Then, according to the rough coordinate of the tooltip, you can find its class name from all top windows. The diagram in Figure 3 illustrates the workflow of inspecting a tooltip, and the section that follows explains how you can use our method.

Figure 3. Workflow of inspecting the tooltip
Inspect tooltip's solution workflow

Use the Test Object Inspector to inspect the base hover object's properties

Figure 4. Get the properties by using the Test Object Inspector
Test Object Inspector, properties view

You can use .class and .id properties to identify the base object. In most cases, the .id property includes a dynamic number. You can use regular expression to express dynamic numbers. Therefore, you can identify this object as Listing 1 shows. You can use the base object's coordinates (266, 307) to estimate the tooltip's coordinates (270, 350 in our example), because the tooltip is close to its base hover object.

Listing 1. Example of finding the base hover object by the properties
RegularExpression idRex = new RegularExpression
    ("view:_id\\d:L:c:recentActivityDataView:\\d+:docTypeIcon\\d+:multiImage\\d",true);
TestObject[] tos = RationalTestScript.getRootTestObject().find(
        SubitemFactory.atDescendant(".class", "Html.IMG",".id", idRex));

The getTopWindows

public IWindow[] getTopWindows()

This returns an array of IWindows. The IWindow interface provides access to the native window layer. This method is never recorded. Any script that uses this method will probably be platform-specific. If there are no top windows (an unlikely situation), a zero-length array is returned.

Print all top windows' class names, heights, and coordinates

Use Rational Functional Tester's getTopWindows API to get all native windows layer, including the tooltip element, as long as it is native. Listing 2 shows an example of the code to use.

Listing 2. Print all top windows' class names, heights, and coordinates
public getPropertyofTooltip()
    {
      IWindow[] thisChildren = RationalTestScript.getTopWindows();
	      for(int i = 0; i < thisChildren.length-1; i++){
	        if(thisChildren[i].toString().indexOf("[x=0,y=0,width=0,height=0]")>0)
		            continue;
	        System.out.println(thisChildren[i].getWindowClassName().toString());
        System.out.println(thisChildren[i].getClippedScreenRectangle().height);
	        System.out.println(thisChildren[i].getClippedScreenRectangle().x);
	   	 System.out.println(thisChildren[i].getClippedScreenRectangle().y);
        }
	    }

Get the class name of the tooltip

You can find out the class name of a tooltip from all top windows, according to the tooltip's fixed height (the height of the example is about 17) and its estimated coordinate (for this example, the estimate is 270, 350, which should be close to the actual coordinate). In the example here, the tooltip's class name is tooltips_class32, and the next window's class name is SunAwtToolkit, For both, the window's height and coordinates are close to the estimated ones, as Figure 5 shows.

Figure 5. Example of getting the properties of the tooltip
screen capture includes class names, coordinates

Take a screen capture of the tooltip

After getting the class name of the tooltip, you can get the rectangle of the tooltip and take a screen capture of it by using Java API robot.createScreenCapture(Rectangle area). Listings 3 and 4 show code examples.

Listing 3. Example of getting the rectangle of the tooltip
  public Rectangle getRect_tooltip(){
    IWindow[] thisChildren = RationalTestScript.getTopWindows();
	    for(int i = 0; i < thisChildren.length-1; i++){
	     if(thisChildren[i].getWindowClassName().equals("tooltips_class32")&&
	     thisChildren[i+1].getWindowClassName().equals("SunAwtToolkit")){   
	     Rectangle r = thisChildren[i].getClippedScreenRectangle();
		     return r;
    }
	   }
	     return null;}
Listing 4. Example of code for the screen capture
  void captureRectangle(String filename, Rectangle area ){
	   try {
	  	BufferedImage capture = null;
	 	Robot robot = new Robot();
			       capture = robot.createScreenCapture(area);
	  	FileOutputStream out = new FileOutputStream(filename);
	 	JPEGImageEncoder encoder = JPEGCodec.createJPEGEncoder(out);
	 	encoder.encode(capture);
	 	out.flush();
	 	out.close();
	 	}catch{
       (Exception e){system.out.println("Error in BitmapOps" + e);}
       }
		     }

      String sTooltipImg = "tooltip.jpg";
      Rectangle rTooltip = this.getRect_tooltip();
      if(rTooltip !=null){
       captureRectangle(sTooltipImg, rTooltip );
      }

How to attach multiple tooltips to one base bitmap image

This section explains how to attach multiple tooltips to one screen capture.

Get a base image

Get the base image's rectangle. Then save a screen capture of the base image (Figure 6 shows an example).

Figure 6. Base image
screen capture

Get the tooltip's image

Using the method described in the previous section, create a screen capture of the tooltip.

Figure 7. Get each tooltip by using this method
Screen capture of names of several tooltips

Attach the tooltip to the base image

  1. It is important to decide where the attachment starts, because the screen capture will be truncated or overlap if the start position is not appropriate. For this example, we used the top-left corner's coordinate offset between the base image and the tooltip as the attachment's start position.
  2. Then attach the tooltip to the base image, pixel by pixel, using the Java API ImageIO, as shown in Listing 5.

Figure 8, which follows the code example, shows the image that results.

Listing 5. Example of code for attaching the tooltip
//Calculate start position of attachment
  int startpositionX = smallRect.x - largeRect.x;
  int startpositionY = smallRect.y - largeRect.y;

 /*** attach the small image smallScreen to large image largeScreen
 * at point(int startpositionX,int startpositionY) in image largeScreen
 * @param smallScreen
 * @param largeScreen
 * @param  startpositionX
 * @param startpositionY
 * @param mergedImage
 * @return  	
 */		
 public static boolean doImageAttach(String smallScreen, String largeScreen,
 int   startpositionX , int startpositionY , String mergedImage){
   Try {
	        BufferedImage largeimage = ImageIO.read(new File
          (LocalSettings.gsResultsImageDir +  largeScreen));
        BufferedImage smallimage = ImageIO.read(new File
          (LocalSettings.gsResultsImageDir + smallScreen)); 	
        File file = new File(LocalSettings.gsResultsImageDir + mergedImage);       
    
        int w1 = largeimage.getWidth();
        int h1 = largeimage.getHeight();
        int w2 = smallimage.getWidth();
        int h2 = smallimage.getHeight();
        int imageType = largeimage.getType();
        BufferedImage imageSaved = new BufferedImage(w1, h1, imageType);
        Graphics2D g2d = imageSaved.createGraphics();
        g2d.drawImage(largeimage, null, 0, 0);

   for (int i = 0; (i < w2)&&(i + startpositionX<w1); i++) {
        for (int j = 0; (j < h2)&&(j + startpositionY<h1); j++) {
	            int _rgb2 = smallimage.getRGB(i, j);
            imageSaved.setRGB(i + startpositionX, j + startpositionY, _rgb2);
          }
     }
   
  if((w2+startpositionX)>w1 || (h2+startpositionY)>h1){
	     Logger.logWarning("the smallSceen to attach to the largeScreen is
         outside the size of the largeScreen!");
     }
   return ImageIO.write(imageSaved, "jpg", file);
	  } catch (Exception e) {
	     
  Logger.logError(e.getMessage());
	    return false;}	}
Figure 8. Resulting bitmap file content, after attachment
screen capture with eight tooltips

Summary

There are two difficulties in attaching multiple tooltips:

  • How to identify the properties of the tooltip, because you can't get them directly by using the Test Object Inspector.
  • Where the proper start position is when adding the attachment, to avoid truncation or overlap.

This article explains the basic methods to solve these problems.


Download

DescriptionNameSize
Source code for the examples in the article1attachtooltip.java7KB

Note

  1. This source code is not a tool that can be used directly for each product. You may need change the code in order to apply this solution to your own product.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Rational software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Rational
ArticleID=932029
ArticleTitle=Use IBM Rational Functional Tester to automatically capture bitmap images of multiple tooltips
publish-date=06042013