UI Automation in the single sign-on world, Part 2: An alternative solution for ISAM ESSO

This two-part series provides a solution for some unsupported single sign-on (SSO) scenarios when deploying IBM® Security Access Manager for Enterprise Single Sign-On (ISAM ESSO). Part 1 discussed SSO architecture and how it fits into different operating system components. This article provides examples of the real work (with sketches) of a few useful ISAM ESSO AccessProfiles based on the UI automation concept. This is an alternative solution and not necessarily the sole, definitive solution.

Oktawian Powązka (Oktawian.Powazka@pl.ibm.com), Software Services Consultant, IBM

Photo of Oktawian PowązkaOktawian Powązka has worked as a services consultant and software engineer for a couple of internal TAM E-SSO IBM projects, mainly in Europe. He is currently working on the TAM E-SSO GIAMA project. Oktawian has four years of experience handling complex solutions in the TAM E-SSO arena.



04 June 2013

Introduction

As mentioned in Part 1 of this series, the current release of IBM Security Access Manager for Enterprise Single Sign-On (ISAM ESSO) doesn't provide any programming interfaces to deliver single sign-on (SSO) solutions using the UI Automation framework. The UI Automation framework predecessor, Microsoft™ Active Accessibility (MSAA), which was added to the profiling API in the early days of TAM ESSO, doesn't fulfill all of the new requirements. Thus, I decided to design a completely new ISAM ESSO plug-in called UIAutomationHelper based purely on the Microsoft UI Automation interface. Please keep in mind that this is an alternative solution, not the singular solution.

Sample environment

To follow along with this article, advanced profiling skills and a lot of patience are recommended. The example assumes that the proper ISAM ESSO environment has already been set up. The following software should be installed and configured on Windows™ XP or Windows 7 (Windows 2003/2008 should also work, but has not been thoroughly tested):

  • IBM Security Access Manager for Enterprise Single Sign-On version 8.2.x. or the former version: TAM E-SSO 8.1.x.
  • For the UI Automation framework, Windows versions below Windows 7 require that a tiny (less than 1MB) Microsoft update be installed on the machine (see Resources). If the update is not installed before running the ISAM ESSO profiled application, the relevant information with the direct link to the Microsoft website will be shown by the AccessProfile during application startup.

UIAutomationHelper high-level architecture

The plug-in was created as a Microsoft COM component accessible from VBScript code. As with any other COM library, UIAutomationHelper needs to be delivered according to the application bitness. There is a separate library dedicated for 32-bit as well as for 64-bit applications as far as 64-bit systems like Windows 7 are concerned.

Figure 1 shows a new child added to the ISAM ESSO components family.

Figure 1. UIAutomationHelper as an extension to the existing ISAM ESSO Plug-ins bundle
ISAM ESSO Plug-ins to integrate itself with other software components

Every COM component must be registered in the Windows registry before it can be used for the first time, particularly when it's going to be called directly from the ISAM ESSO AccessProfile VBScript code using the CreateObject function. In actuality, when CreateObject is invoked Windows will look up the registry to find out how the object should be instantiated. Windows will then try to load the corresponding library (UIAutomationHelper[32/64].dll) and create the object from it.

I've prepared two simple registry files to set up proper Windows registry entries. As with the plug-in itself, there are two such files you can download (for the corresponding operating system version):

  • UIAutomationHelper32.reg, designed to be run on 32-bit systems like Windows XP
  • UIAutomationHelper64.reg, designed to be run on 64-bit systems like Windows 7

For simplicity's sake, I have not specified any precise path as far as the plug-in library is concerned. The easiest method is to directly copy UIAutomationHelper[32/64].dll (download) into the C:\Windows directory. Or, you could modify the UIAutomationHelper[32/64].reg files and specify an extra directory path before the library name. (If you have any file system or registry problems, it is suggested you use Process Monitor (see Resources) to investigate the root cause of the problem.)

The next section explores some of the more technical aspects of the UIAutomationHelper plug-in architecture and internals. (This knowledge is definitely not required to write AccessProfiles based on the plug-in, so you can safely skip the following section if so inclined.)

The plug-in internals

Because the UIAutomationHelper plug-in is designed as a COM component, it needs to comply with certain Microsoft COM architecture criteria. Though this topic goes beyond the scope of this article, following are some code samples that are closely related to the VBScript counterpart in the AccessProfile. For example, the number 006738ED-1D1F-455F-AC08-3F65CCA71BAD is a 16-byte (128-bit) array that is filled in with a unique series of bytes and is called GUID. In the realm of the COM world, it's used to uniquely distinguish different software component interfaces. Almost every Windows user is probably more than aware of those peculiar GUID values flowing around the registry.

Using a simplified model, when the VBScript code executes the CreateObject function using UIAutomationHelper.dll as an argument, the call is indirectly transferred into the following Windows registry entry:
[HKEY_CLASSES_ROOT\CLSID\{006738ED-1D1F-455F-AC08-3F65CCA71BAD}\InprocServer32]

This entry points out the right library file implementing the given COM interface.

The second relevant part, as far as the AccessProfile functionality is concerned, is the UIAutomationHelper plug-in methods. I've designed nine such methods to comply with the typical AccessProfile flow. Listing 1 shows all nine methods.

Listing 1. COM related aspects of the UIAutomationHelper plug-in
...
interface ICOMServer : IDispatch
{
     [id(1)] HRESULT Init                    ([in] LONG HWND, [out,retval] LONG* pRet);
     [id(2)] HRESULT ReleaseUI               (void);
     [id(3)] HRESULT EnableLog               ([in] BSTR FileName);
     [id(4)] HRESULT IdentifyClickedControl  ([in] BOOL Mouse, [out,retval] BSTR*
        pRet);
     [id(5)] HRESULT BuildScreenStruct       ([in] BSTR sName, [out,retval] LONG* pRet);
     [id(6)] HRESULT FindUIControl           ([in] BSTR sElemName, [out,retval] 
        BOOL* pRet);
     [id(7)] HRESULT GetText                 ([in] BSTR sElemName, [out,retval] BSTR*
        sName);
     [id(8)] HRESULT SetText                 ([in] BSTR sElemName, [in] BSTR sName);
     [id(9)] HRESULT Click                   ([in] BSTR sElemName);
}
[ uuid(006738ED-1D1F-455F-AC08-3F65CCA71BAD),
]
...

Let's unravel some of the internals surrounding the methods in Listing 1.

Init
Initializes the UIAutomationHelper plug-in. It needs to be called just once, somewhere during the ISAM ESSO AccessProfile startup phase. With Init:
  • If the window handle is not passed as an argument, this method identifies the active foreground window by iterating through all sub-windows searching for one that has one of the following classes:
    • MicrosoftSilverlight (Silverlight)
    • HwndWrapper[ (Windows Presentation Foundation)
    • MacromediaFlashPlayerActiveX (Flash/Flex from Adobe)
    These are the three window's classes that have been thoroughly tested and are supported by the plug-in. (Many more UI interfaces are supported by the UI Automation framework but I did not have time to test them.) In cases where the window handle argument is given, it needs to be a window that directly embeds Flash/Flex or Silverlight/WPF content.
  • Having found a matching window, its handle is used to install a Hook (monitoring procedure) into the application message-handling mechanism. (This provides sort of an auxiliary method to the existing Microsoft UI Automation framework regarding capture of sensitive information such as passwords. The main concept behind this is discussed below.)
  • A separate thread is created to initialize and create an instance of the UIAutomation core object based on the window handle obtained during the previous phase. This has the following implications:
    • From this point, the UIAutomation object is kept initialized as long as the AccessProfile application is running or until AccessProfile calls the ReleaseUI method.
    • All subsequent calls into every new AccessProfile VBScript instance are reusing the already-instantiated UIAutomation object and its dedicated thread.
    The design keeps the ISAM ESSO and application symbiosis as smooth as possible. To keep plug-in responsiveness at the highest possible level, the VBScript invocation thread is completely separated from the UIAutomationHelper core thread. Inter-thread communication is done using a message-handling mechanism.
ReleaseUI
Frees the UIAutomation object and all interrelated elements. This method may be used after the SSO procedure is no longer needed, especially in rare cases when a performance impact due to UI Automation framework usage is noticeable. Technically speaking, ReleaseUI is not mandatory because all UI Automation interrelated objects are closed and freed when the application terminates.
EnableLog
The log name is defined by the FileName argument. By default, logs are saved in the current User Profile folder (pointed out by the %USERPROFILE% env. variable, usually the user's home directory). Log structure is discussed in UIAutomationHelper logging.
IdentifyClickedControl
Has a two-fold usage:
  • During the AccessProfile build-up phase, it helps to identify all UI controls visible on the screen, as in Figure 2. (The meaning of those UI identifier buzzwords is explained below.) This UI element identifier will be reused in other methods.
  • During the period of normal application flow, it helps to identify the selected (either by mouse or keyboard) UI element.
Such selection is later compared with the value pre-saved during the AccessProfile build-up phase.
BuildScreenStruct
The core method responsible for rebuilding the information about all visible UI elements such as windows and controls. BuildScreenStruct does have some performance impact and, therefore, should be used prudently. Obviously, your given application influences when and how frequently this method should be called. See Example.
FindUIControl
A counterpart of the "window is found" or "html element is found" trigger of the ISAM ESSO profiling API.
GetText
A counterpart of the "capture credentials" action of the ISAM ESSO profiling API.
SetText
A counterpart of the "inject credentials" action of the ISAM ESSO profiling API.
Click
A counterpart of the "Click a window" or "Click a web element" action of the ISAM ESSO profiling API.
Figure 2. IdentifyClickedControl called during AccessProfile build-up phase
UI frameworks relationship

Example

In the following example, you will create an AccessProfile for a web application based entirely on Silverlight technology. Actually, the example application resembles the whole operating system accessed through a web page and, thus, presents quite a challenge regarding performance-related aspects. When a user passes the logon screen and gets to the main desktop, there are hundreds of distinctive UI elements that can be potentially placed on the desktop. All of those elements need to be traced if you want to reliably monitor and capture the logoff sequence. Of course, if the re-login scenario is not an issue the whole AccessProfile would be a piece of cake.

silveos.com belongs to the group of websites that have windowless mode enabled. (See Part 1 for a discussion of the windowless concept.) What's needed at the AccessProfile entry point is the VBScript designed to disable this windowless mode attribute.

For the sake of an argument, the easiest way to check if windowless mode is enabled is to use AccessStudio and verify the website content signature. No VBScript is required when a signature does have a reference to the Silverlight object window, as in the example in Figure 3. When windowless mode is disabled, AccessStudio is able to construct the proper signature for Silverlight content.

Figure 3. Windowless mode is disabled
ISAM ESSO Plug-ins to integrate itself with other software components

There are several steps required to create a fully-fledged ISAM ESSO AccessProfile for the example web application. In Figure 4, I've numbered all the places where a dedicated VBScript is used to call some UIAutomationHelper methods. Although visually this example doesn't depart from any other really trivial AccessProfile case, all the nitty-gritty time consuming details are buried inside those VBScripts… you have been warned.

Figure 4. Complete ISAM ESSO AccessProfile schema designed for SilveOS website
ISAM ESSO Plug-ins to integrate itself with other software components

As with any other irregular profiling use case, the easiest way to start is to copy the existing AccessProfile artifact and use it as a template. However, please do not consider this template as an absolute. For example, this whole profile can be condensed into half of the existing size.

Just as a reminder, after the AccessProfile is fully loaded based on the document complete event, the entry point of that profile must contain "Web page completes loading" as the first trigger that initiates the so-called state machine. Assuming this is the case, the first element to change is a "Web page completes loading" trigger. This trigger's signature needs to match that particular website content. Figure 5 shows an example.

Figure 5. "Web page completes loading" trigger's signature
ISAM ESSO Plug-ins to integrate itself with other software components

You'll also use that trigger to call the VBScript designed to disable the windowless mode attribute (number 1 in Figure 4). The details of that particular VBScript structure are thoroughly explained in Part 1.

After a little pause, providing time to reload the Silverlight object back into the browser and instantiate the dedicated window for this, you pass through another VBScript (number 2 in Figure 4). Listing 2 shows the VBScript content.

Listing 2. VBScript designed for UIAutomationHelper plug-in initialization
 1 On Error Resume Next
 2 Set oShell = CreateObject("Wscript.Shell")
 3 Set pc  = runtime.GetPropertiesContainer()
 4 appName = runtime.GetAccessProfileId()
 5
 6 pc.SetPropValue "UserID",       "TypeID:50004#Index:10"
 7 pc.SetPropValue "PwdID",        "TypeID:50004#AutomationId:passwordBox"
 8 pc.SetPropValue "LoginButton",  "TypeID:50000#AutomationId:btnLogin"
 9 pc.SetPropValue "LogOutButton", "TypeID:50000#AutomationId:btnlock"
10 pc.SetPropValue "CloseButton",  "TypeID:50000#AutomationId:CloseButton"
11 pc.SetPropValue "LoginError",   "TypeID:50020#Name:The username or password is 
      incorrect"
12
13 pc.SetPropValue "ActiveScreen", ""
14 pc.SetPropValue "ControlID", -1
15 pc.SetPropValue "Error", 0
16
17 appName = runtime.GetAccessProfileId()
18 Err.Clear
19 Set UIAutomationHelper = CreateObject("UIAutomationHelper")
20 If (Err=0) Then
21    bRet=UIAutomationHelper.Init(0)
22    If Err=0 And bRet=0 Then
23       UIAutomationHelper.EnableLog "UIAutomationHelper.log"
24    ElseIf (bRet=-1) Then
25       button = Msgbox ("Please install UIAutomation component from following 
            location:"&vbNewLine&"http://support.microsoft.com/kb/971513", 
            vbYesNo+vbCritical+vbSystemModal, "UIAutomation Error")
26       If button = vbYes then oShell.Run ("http://support.microsoft.com/kb/971513")
27       pc.SetPropValue "Error", 1
28    ElseIf (bRet=-2) Then
29       MsgBox "Window Initialization Error."&vbNewLine&"If this is 
            Flash/Flex/Silverlight website please make sure to add the script to setup 
            the Window Mode property", vbExclamation+vbSystemModal, appName
30       pc.SetPropValue "Error", 1
31    End If
32 Else
33    MsgBox "UIAutomationHelper Initialization Error."&vbNewLine&"Please make sure to 
         run UIAutomationHelper.reg file to register UIAutomationHelper component", 
         vbExclamation+vbSystemModal, appName
34    pc.SetPropValue "Error", 1
35 End If

To keep things brief, I'll omit obvious parts clearly related to the ISAM ESSO profiling API itself. Table 1 explains the code lines in Listing 2.

LinesDescription
Lines 6 to 11Depict all the relevant UI elements as far as the SSO procedure is concerned. Those values have been gathered during the AccessProfile build-up phase by the IdentifyClickedControl method. Figure 2 showed a use case example for which I'll describe the required steps.
Line 19Creates a UIAutomationHelper COM object. Most of the failures during UIAutomationHelper instantiation are caused by missing Windows Registry entries or UIAutomationHelper[32/64].dll library inaccessibility (see UIAutomationHelper high-level architecture).
Line 21Initializes the UIAutomationHelper plug-in. The initialization process happens just once within the whole profile, but it always needs to be called before any other UIAutomationHelper related method.

There is one argument to be used for this method: a window handle (HWND). If it's 0, UIAutomationHelper uses the currently active foreground window. Otherwise, the profile can pass a handle using the GetHWNDFromXPath method, as in the following example:

GetHWNDFromXPath ("/child::wnd[@title#""Spark Intranet Sample App.*"" and
@class_name=""IEFrame""]/descendent::wnd[@class_name=""MacromediaFlashPlayerActiveX""]")

Keep in mind that in cases where the window handle argument is given it needs to be a window that directly embeds Flash/Flex or Silverlight/WPF content.

Line 23 Enables the logging mechanism used for some deeper GUI interface structure investigation. Log files are saved in the current User's Profile folder (user's home directory). Log structure is discussed in UIAutomationHelper logging.

I defined two separate UIAutomationHelper initialization errors. If the error is caused by a missing Microsoft update, an informative message points the user to the direct link in the Microsoft website. The other error is related to the windowless mode issue.

You've passed through the second VBScript and now have the option to manually change the profile flow and go back into the UI interface build-up phase. This lets you revalidate all UI controls associated with the SSO process. The profile needs to be restructured. After restarting the AccessStudio Test mode, the profile lets you identify each subsequent UI control just by clicking on it.

The easiest way to gather the UI identifier buzzword is to press the CTRL-C sequence on the UI Element identifier window and paste it into your favorite text editor. The relevant portion needs to be extracted according to the examples in Listing 2. Yes, it's bizzarre, and perhaps this is an area where improvements in the form of a nice GUI interface would be in order. Nevertheless, capturing those six UI identifiers should not take more than a couple of minutes.

There is one more viable aspect I need to add in the UI element identification procedure. When you look closely at the six UI control's signatures in Listing 2, the first one (for UserID) is somewhat out of the scope when compared with the other five. Figure 6 shows this UI identifier part captured just after the mouse click.

Figure 6. UserID captured during AccessProfile build-up phase
ISAM ESSO Plug-ins to integrate itself with other software components

The Name attribute, which most of the time should be either empty or have some meaningful predefined static entry, in this case, contains the value entered by the user into the editable field itself. Obviously, the Guest value can't be used for any possible identifier. That is one reason why, for all UI Element identifiers, I've used as many as four possible tags to have a better chance to uniquely distinguish each relevant UI control. The four tags are:

TypeID
Used to describe UI control type. For example, 50004 denotes Edit Control, 50000 denotes Button Control, and so on.
AutomationId
Contains the unique UIAutomation identifier (ID) for every automation element (unfortunately, as shown in the last example, it's not always defined).
Name
The text representation of the UIAutomation element. This entry should indicate some kind of descriptive label.
Index
Used to assign a record number into every subsequent control participating in the visual representation of the whole UI structure. Unfortunately, the record numbers change when new UI elements (like pop-up windows) are added to the mix. Due to the unpredictability of the record number, the Index part of the identifier should only be used as the last resort. Though using Index works perfectly in this particular login dialog case, it must always be thoroughly tested before it can be assigned into the AccessProfile.

The tags above are prioritized in exactly the same order. If AutomationId has not been assigned, the next one to be used is the Name tag. The Index tag will be used as the last resort.

After passing through 20 or 30 different websites based on the Flash/Flex or Silverlight frameworks, I found about 10% that were completely messed up regarding UI Automation and Active Accessibility information. (Usually, Flash/Flex is based on MSAA implementation proxied by the UI Automation framework.) That 10% was absolutely out of the scope as far as SSO functionality is concerned. Of course, the main issue was the inability to uniquely identify UI elements. Of the 90% of achievable cases, some do require a lot of testing and troubleshooting during the profile creation period. The whole UI element's identification mess reminds me of other bizarre GUIs, such as LabVIEW, which are (to some extent) based on the standard Win32 native UI API.

Back to my example. After I cleared the existing user name value and pointed into the same field again with the predefined priority, Index has been picked up as the next possible identifier tag, as shown in Figure 7.

Figure 7. UserID part captured again after clearing up the user name value
UI element identifier window with the TypeID defined.

This identifier, though not fully unique, works perfectly in the case of this particular login window. I haven't encountered any other serious issues when passing through the rest of the UI controls on this screen. The next step is to characterize the logoff sequence.

To start, you need to search for a Logout button. As already noted, when a mouse is being clicked, the UI Element identifier dialog "swallows" the mouse event. Therefore, to pass through the whole login screen and get into the next screen (main desktop) AccessAgent needs to be temporarily disabled and enabled again when a relevant UI object is visible. Figure 8 shows the next phase, which is the Logout button identifier captured on the main desktop.

Figure 8. Logout button identifier
ISAM ESSO Plug-ins to integrate itself with other software components

Luckily, the Logout button can be identified by AutomationId, which is a type of unique identifier to the UI Automation framework.

Having passed through the UI interface build-up phase, you can now go back and restructure AccessProfile back into the "working" state. Number 3 in Figure 4 indicates the next VBScript in line to be processed. Listing 3 below shows the contents of that VBScript.

Listing 3. Clear up the UserID field
1 On Error Resume Next
2 set pc = runtime.GetPropertiesContainer()
3
4 Set UIAutomationHelper = CreateObject("UIAutomationHelper")
5 bRet = UIAutomationHelper.BuildScreenStruct ("")
6 If Err=0 And bRet=0 Then
7    UIAutomationHelper.SetText pc.GetPropValue ("UserID"), ""
8 End If

To keep things brief, parts that are repeatable in every script and parts clearly related to the ISAM ESSO profiling API itself aren't discussed.

LineDescription
Line 5Shows the method responsible for rebuilding the information about all visible UI elements such as windows and controls. This constitutes an entry point to other UIAutomationHelper methods related to finding, getting, or setting values on the screen. Put simply, BuildScreenStruct needs to be used every time there's a chance that something has been changed concerning the UI structure. A few more examples will clear this up.
Line 7Is used to set an empty string value into the UserID field. (It's assumed that the corresponding UI element identifier has already been assigned; if this is unclear, look again at Listing 2.) This predefined Guest entry is already filled in as the user name value.

The next state, number 4 from Figure 4, involves the VBScript in Listing 4, which is called periodically with the rate of one call per second. As you might guess, this is used as a sort of monitoring sequence.

Listing 4. Is the login screen ready?
 1 On Error Resume Next
 2 set pc = runtime.GetPropertiesContainer()
 3
 4 Set UIAutomationHelper = CreateObject("UIAutomationHelper")
 5 bRet = UIAutomationHelper.BuildScreenStruct ("")
 6 If Err=0 And bRet=0 Then
 7    If (UIAutomationHelper.FindUIControl (pc.GetPropValue ("LoginError"))=0 And 
         UIAutomationHelper.FindUIControl (pc.GetPropValue ("LoginButton"))=1) Then
 8       pc.SetPropValue "ActiveScreen", "Logon"
 9    End If
10 End If

The script in Listing 4 contains only one new sequence not discussed thus far. Line 7 is used to check if the so-called LoginButton (shown in Figure 2) is displayed and if there are no visible error messages, as in Figure 9. If this condition is fulfilled, AccessProfile continues its flow.

Figure 9. LoginError property
A username or password is incorrect error message.

From this point on, the Login screen is ready for credentials injection (Number 5 in Figure 4). The VBScript in Listing 5 shows this case.

Listing 5. Injects credentials into the login screen
 1 On Error Resume Next
 2 set pc = runtime.GetPropertiesContainer()
 3 Set udp = runtime.GetUserDataProvider()
 4
 5 pc.SetPropValue "ControlID", ""
 6 Set UIAutomationHelper = CreateObject("UIAutomationHelper")
 7 If(Err=0) Then
 8    if (pc.IsAccDataBagItemExist ("inject_bag", "aditi_ciuser") And 
         pc.IsAccDataBagItemExist ("inject_bag", "aditi_cspwd")) Then
 9       User     = pc.getAccDataItem("inject_bag", "aditi_ciuser")
10       Password = pc.getAccDataItem("inject_bag", "aditi_cspwd")
11       UIAutomationHelper.SetText pc.GetPropValue ("UserID"), User
12       UIAutomationHelper.SetText pc.GetPropValue ("PwdID"), Password
13       authSvcID = pc.GetAccDataBagAuthId ("inject_bag")
14       saInjectionPolicy = udp.GetAuthPolicy (authSvcID, 
            "pid_auth_inject_pwd_entry_option")
15       pc.SetPropValue "InjectionPolicy", saInjectionPolicy(0)
16       If saInjectionPolicy(0) = 1 Then                          'autologon = 1, 
            always = 2, ask = 3, never = 4
17          pc.SetPropValue "ControlID", "Login"
18          UIAutomationHelper.Click (pc.GetPropValue("LoginButton"))
19       End If
20    Else
21       pc.SetPropValue "Info", "Empty Wallet"
22    End If
23 End If

In Listing 5, most of the elements are clearly related to the ISAM ESSO profiling API. The only new sequences concerning the UIAutomationHelper are in lines 11, 12, and 18. The SetText method is used to inject a string value into the user name and password fields (again, those UI control signatures have been preassigned in Listing 2). Line 18 simulates a mouse click with the Click method, which is a counterpart of the "Click a window" or "Click a web element" action of the ISAM ESSO profiling API.

Obviously, the automatic mouse click simulation should only happen when there is a corresponding entry in the wallet (password entry option is set to Automatic Logon). Again, this is a counterpart of selecting Yes in the "Advanced Options / Execute only if autologon is enabled" field.

Listing 6 shows the script (Number 6 from Figure 4) performing capture of the user name and password entries when a mouse click occurs. The profile stops on this particular state when either: there have been no credentials in the wallet or the password entry option has been set to a value different than Automatic Logon. To some extent, this is a counterpart of the capture credentials action of the ISAM ESSO profiling API, although in this case, the entered values are saved into two local properties.

Listing 6. Store user name and password
 1 On Error Resume Next
 2 set pc = runtime.GetPropertiesContainer()
 3 Set udp = runtime.GetUserDataProvider()
 4
 5 Set UIAutomationHelper = CreateObject("UIAutomationHelper")
 6 If (Err=0) Then
 7    ControlID = UIAutomationHelper.IdentifyClickedControl(1) 'Mouse
 8    pc.SetPropValue "ControlID", ControlID
 9    If (ControlID = pc.GetPropValue ("LoginButton")) Then
10       User       = UIAutomationHelper.GetText(pc.GetPropValue ("UserID"))
11       Password   = UIAutomationHelper.GetText(pc.GetPropValue ("PwdID"))
12       pc.SetPropValue "User", User
13       pc.SetPropValue "Password", Password
14       If (User <> "") Then pc.SetPropValue "ControlID", "Login"
15    End If
16 End If

Listing 6 contains two new methods not discussed thus far.

LineDescription
Line 7Checks which UI control has been clicked by the user. Capture credentials should only happen when an already-identified button (red-boxed element from Figure 2) has been clicked. The IdentifyClickedControl method has two possible arguments:
  • 1 denotes the checks for mouse clicks.
  • 0 denotes the checks for keyboard press.
The UIAutomationHelper plug-in itself doesn't follow all of the user's moves, which is why it needs to be advised if the last event was the mouse click or if a key was pressed. This particular web application seems to ignore usage of the Enter key, which is why there's no need to capture it within the profile.
Lines 10-11Receive the user name and password entries from the corresponding UI elements.

If credential capture did take place, the right condition is fulfilled and the profile continues its flow. VBScript in Listing 7 shows the next case (Number 7 from Figure 4).

Listing 7. If there are no errors, copy credentials into dedicated bag
 1 On Error Resume Next
 2 set pc = runtime.GetPropertiesContainer()
 3
 4 Set UIAutomationHelper = CreateObject("UIAutomationHelper")
 5 If (Err=0) Then
 6    bRet = UIAutomationHelper.BuildScreenStruct ("")
 7    If Err=0 And bRet=0 Then
 8       bRet = UIAutomationHelper.FindUIControl (pc.GetPropValue ("LoginError"))
 9       If CBool(bRet)=True Then
10             pc.SetPropValue "ActiveScreen", "LoginScreen"
11          Else
12             pc.SetPropValue "ActiveScreen", "MainScreen"
13             pc.SetAccDataItem "capture_bag", "aditi_ciuser", pc.GetPropValue ("User")
14             pc.SetAccDataItem "capture_bag", "aditi_cspwd",  pc.GetPropValue 
                  ("Password")
15          End If
16    End If
17 End If

Listing 7 doesn't contain anything new as far as UIAutomationHelper functionality is concerned. It merely checks if there is an error message due to an incorrect user name or password, as in Figure 9. If there is an error message, it switches the profile flow back into the login screen. Otherwise, it copies the user name and password values into a dedicated "bag" and subsequently, under the next trigger, saves credentials into the wallet.

The Login screen sequence has already been presented, so the last part to discuss is the main desktop. The only sequence you need to monitor is the logoff procedure, which basically means waiting for the Logout button to be clicked. The VBScript in Listing 8 shows the last case (number 8 in Figure 4).

Listing 8. Waiting for Logout button to be clicked
 1 On Error Resume Next
 2 set pc = runtime.GetPropertiesContainer()
 3 Set udp = runtime.GetUserDataProvider()
 4
 5 Set UIAutomationHelper = CreateObject("UIAutomationHelper")
 6 If (Err=0) Then
 7    ControlID = UIAutomationHelper.IdentifyClickedControl(1) 'Mouse
 8    If (ControlID = pc.GetPropValue ("LogOutButton")) Then
 9       pc.SetPropValue "ControlID", "LogOut"
10    End If
11 End If

Line 7 checks which UI control has been clicked by the user. Again, you only check the mouse case—no keyboard. After the Logout button has been clicked, the profile goes back into the Login screen and monitors any possible re-logon cases.

It's important to note that in the VBScript in Listing 8 I haven't used the BuildScreenStruct method before recognizing which control has been clicked. As mentioned, typically that method should be used every time there is a chance that something has been changed regarding the UI structure. Yet, this does have some performance impact and should be used prudently. It looks like this particular Silverlight application does build up the matrix of most desktop UI elements.


UIAutomationHelper logging

Listing 9 shows a brief excerpt from a log file created by the EnableLog method when analyzing the Silverlight application example. The listings tend to be huge so I've selected only some of the final lines.

Listing 9. UIAutomationHelper log file
 1 14:
 2 TypeID:50000 (Button Control   ), AutomationId:btnLogin , Name:
 3 15:
 4 TypeID:50006 (Image Control    ), AutomationId: , Name:
 5 16:
 6 TypeID:50020 (Text Control     ), AutomationId:tbNotRegistered , Name:You can use
 7 17:
 8 TypeID:50005 (Hyperlink Control), AutomationId:hlGuest , Name:Guest
 9 18:
10 TypeID:50020 (Text Control     ), AutomationId:tbNotRegisteredEnd , Name:account or 
      register
11 19:
12 TypeID:50005 (Hyperlink Control), AutomationId:hlRegisterNow , Name:Register now
13 20:
14 TypeID:50020 (Text Control     ), AutomationId:tbRegistered , Name:Already registered?
15 21:
16 TypeID:50005 (Hyperlink Control), AutomationId:hlLogin , Name:Back to Login
17 22:
18 TypeID:50000 (Button Control   ), AutomationId:btnRegister , Name:OK
19 23:
20 TypeID:50006 (Image Control    ), AutomationId: , Name:
21 24:
22 TypeID:50020 (Text Control     ), AutomationId:tbDisclaimer , Name:This is 
      demonstration site for open source community. Project has no relationship to 
      Microsoft corporation.
23 ---------------------------------------------------------------------
24 Following Item has been clicked:
25 TypeID:50004 (Edit Control     ), AutomationId: , Name:
26 TypeID:50004#Index:10
27 Following Item has been clicked:
28 TypeID:50006 (Image Control    ), AutomationId: , Name:
29 TypeID:50000#AutomationId:btnLogin
30 GetText
31 TypeID:50004 (Edit Control     ), AutomationId: , Name:Guest
32 Guest
33 GetText
34 TypeID:50004 (Edit Control     ), AutomationId:passwordBox , Name:
35 password

Table 4 lists, step by step, what the log structure in Listing 9 is all about.

LineDescription
Lines 1 to 22Show some of the UI controls on the login window already presented in many places in this article. Each control is defined by: Index starting at line 14; TypeID, which helps to define UI control purpose; AutomationId for unique identifier; and Name. As noted earlier, the AutomationId and Name attributes are somewhat optional and are not assigned to some of the visible UI elements.
Lines 24 to 29Show two cases of mouse clicks. The first shows clicking the User Name field, which is described neither by AutomationId nor the Name, thus the final signature has been defined using the Index tag (TypeID:50004#Index:10).

The second case looks like clicking the Login button, although not directly. In reality, on the screen there are two UI elements—one covering the other. Those two elements have exactly the same coordinates. The problem is that from the SSO point of view you usually care about button control not image control. In accordance with this assumption, I used parent-child relationships to swap to always gather a Control that is more important as far as SSO is concerned. To summarize, in this case, the profile always picks up button over image if the latter fully covers the former (thus, the identifier is TypeID:50000#AutomationId:btnLogin).

Lines 30 to 35 Show two cases of calling the GetText method. There are two lines dedicated per each call: the first line shows the UI element identifier description, and the second shows the captured value.

To be continued...

I was initially planning to present four profiling cases as examples of UIAutomationHelper usage. But, due to time and space concerns, I'll leave the remaining three examples for a future publication. In the meantime, you can download the whole package encapsulating all four profiles. The three not covered in this article are (see Resources for links):

  • SparkIntranetSample: Flex sample application, an example dedicated for a web application written using the Flex framework.
  • Logon Screen Demo: An interesting logon screen example based on the WPF Framework.
  • visualspicer: A decent piece of Flash framework sample.

As mentioned, about 90% of cases are potentially profileable in terms of the UI Automation framework and SSO solutions. Hopefully, you will find the examples to be a suitable alliance.


Conclusion

Without a doubt, UI Automation seems to be a very helpful and supportive technology. Yet, as is often the case, it has its problems, as follows.

  • There is some performance impact in addition to ISAM ESSO itself. (SSO is definitely one of those software components that impose a bit of overhead regarding performance.)
  • As demonstrated in this article, a UI element's property value, and Name and Index information, are not quite reliable. For example, the visibility attribute (in the Silverlight application case) is very misleading and can't be relied upon. Similarly, from time to time the Name attribute gets overwritten by a random value or is not set up at all. In this context, the most reliable attribute as far as identification goes is AutomationId, which, unfortunately, is not always assigned. A potential solution for this conundrum involves building some kind of descriptive matrix based on the UI element's parent-child relationship.
  • As far as I can tell, fields containing sensitive information like passwords are protected and can't be directly enquired. In this sense, the UI Automation framework is slightly different than the native Win32 API, which doesn't impose such restrictions. To be able to read any Win32 native control (containing sensitive information or not) it's just enough to keep interrogating code within the process boundary. Due to this restriction, I was forced to add some auxiliary methods to the existing Microsoft UI Automation framework regarding capture of the sensitive information.

Considering the whole concept, I'd like to draw two final conclusions:

  • Use of the UI Automation framework as a helper method to the existing ISAM ESSO functions is definitely reasonable in cases where a corresponding application doesn't have excessive UI design (performance impact) and has some reliable accessibility information embedded into its design.
  • Although calling the UI Automation framework indirectly with a VBScript is definitely doable (as proven in this article), it would be more user friendly and robust to integrate this function within the ISAM ESSO native engine.

Downloads

DescriptionNameSize
ISAM ESSO profiles bundle:UIAutomation_Bundle.eas101KB
Windows registry file for COM component (32bit)UIAutomationHelper32.reg2KB
Windows registry file for COM component (64bit)UIAutomationHelper64.reg3KB
UIAutomationHelper plug-in library (32bit)UIAutomationHelper32.dll85KB
UIAutomationHelper plug-in library (64bit)UIAutomationHelper64.dll30KB

Resources

Learn

Get products and technologies

  • Evaluate IBM products in the way that suits you best: Download a product trial, try a product online, or use a product in a cloud environment.

Discuss

  • Get involved in the developerWorks community. Connect with other developerWorks users while exploring the developer-driven blogs, forums, groups, and wikis.

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Security on developerWorks


  • Bluemix Developers Community

    Get samples, articles, product docs, and community resources to help build, deploy, and manage your cloud apps.

  • Security

    Pragmatic, intelligent, risk-based IT Security practices.

  • DevOps Services

    Software development in the cloud. Register today to create a project.

  • IBM evaluation software

    Evaluate IBM software and solutions, and transform challenges into opportunities.

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Security
ArticleID=932194
ArticleTitle=UI Automation in the single sign-on world, Part 2: An alternative solution for ISAM ESSO
publish-date=06042013