I recently visited Andy Updegrove’s blog, following our IAccessible2 (IA2) announcement, and I noticed his question regarding why we chose to create an API other than UI Automation (UIA). I have had similar questions asked both publicly and privately. The answer to that begins in March 2005.
At CSUN 2005, I approached Microsoft with the idea of producing an industry standard, unencumbered, royalty free, accessibility API. Industry needed a comprehensive, standard, accessibility API to support solid interoperability between applications and assistive technologies. We had a 2 pronged strategy. Microsoft, I was sure, would push for the API being UIA and if this was not successful our alternative was a new accessibility API we were creating out of project Missouri that you all know now as IAccessible2.
At this time, the UNIX accessibility efforts were such that a new API was tenable although we would have, and did, take a lot of heat for suggesting it. That said, we felt it worth the effort to make this happen. We were willing to use UIA even though we had reservations which I discuss later in this blog entry. An indication that this effort was going on can be seen in a June 2005 Microsoft announcement. By October 2005, we were unable to make this happen and we had to cut and run because it would hold up our UNIX accessibility efforts in the Free Standards Group (FSG) as well as our new office suite enablement work for Workplace.
At this juncture I told Microsoft that we would take an alternative approach to achieve the goals I outlined above. Shortly after, the State of Massachusetts laid down the aggressive timeline for ODF accessibility and Missouri kicked into high gear across IBM. If we had stuck with UI Automation we would not have met the states needs, as you will soon read. Also, through industry contribution, UNIX accessibility had now matured significantly and going back to UIA would be similar to a slash and burn effect for that platform. Apple was in a similar situation. Fortunately, in the background we had been creating IA2. The goals of that project are defined in my previous blog entry.
My reservations about UI Automation then and today are multi-faceted and this is a result of a similar situation which happened when Microsoft released Microsoft Active Accessibility (MSAA) in 1995. So, let me digress:
MSAA (formerly OLE Accessibility) was designed with Assistive Technology Vendors (ATVs) input but with little contribution by the developer community creating a chicken and egg problem. Upon release it had little to no ATV support because application vendors did not implement it. This was because ATVs did not support it and because very little guidance was given to developers on how to implement it. As a result, MSAA had great promise but suffered from a number of gaping holes such as: support for rich text; tables (spreadsheets, etc.); hypertext links; relationships; and multiple named actions. Over the years industry vendors have produce a plethora of proprietary APIs, including Microsoft, to expose the information needed to fill the holes. ATVs have had to aggregate support for all these APIs and less reliable, dated, engineering techniques like “screen scraping” to produce an offscreen model. The result has been a nightmare for Assistive Technology Vendors (ATVs) and application developers to support and the results are often unreliable. In fact, one could argue that no single, comprehensive, standard, accessibility API for Windows exists.
Microsoft developed UI Automation with the intention of filling these holes but many similar problems exist today. Today, assistive technology participation in UI Automation appears to be limited to design time. Currently, the major ATVs don’t support UI Automation (UIA).
UIA is divided into a provider(application) and client side (ATV) API. The client side portion runs in managed code called the Common Lanugage Runtime environment. Today UIA requires ATVs to rewrite all or part of their software to run in a managed environment to access them. This has prevented widespread ATV adoption of UIA and thus application vendor support. Microsoft needs an unmanaged client-side UIA implementation, which ATVs can support, but that is not currently available for Vista or XP. Given the absence of an unmanaged UIA API and the extremely few applications that support UIA, we believe UIA really needs more ATV testing to prove readiness for wide scale adoption. Microsoft's challenge will be to get large applications, like MS Office, and ATVs to support UIA to work out any defects.
Now, all this said, the Vista desktop needed to be accessible. Vista access by screen readers is a combination of MSAA and screen scraping. In short, access to the Vista desktop is like XP today. Screen scraping is limited to the unmanaged environment. For managed applications Microsoft provided some basic UIA to MSAA mapping and some text access but the accessibility is limited. Consequently, support for managed applications is currently dependent up on a limited client side unmanaged API layer. Due to the limited number of managed applications, there is little business justification for ATVs to support them.
Getting back to IA2, I will turn the clock bat to 1997 when IBM and Sun got together to collaborate on a new accessibility API for Java – now called the Java Accessibility API. You can learn about the early work in my 1998 CSUN pitch with Phill Jenkins. The presentation includes a link to guidelines I created to assist developers. A new Java Accessibility API was needed because of MSAA’s dependence on other technologies, like offscreen models, which were not available on other platforms. Additionally, IBM created a pure Java Screen Reader called the Self Voicing Kit for Java to test the implementation on Swing with a real, scriptable, screen reader. This work is something both Sun and IBM are very proud of as it is also the basis for the work on UNIX, UNO, and IA2. The API works. Java’s accessibility problems today are not with the API but the transport layer in the Java Access Bridge. The use of MSAA and IA2 would be an excellent replacement.
We do believe that more work is required for application vendors to use UIA. With IA2 we complement MSAA, as is, and we have fully tested ATV support today. To validate this I refer to GW Micro's endorsement of IAccessible2:
"MSAA provides a great base for allowing applications to make themselves accessible. However, over the years it has become apparent that MSAA lacks important information needed to make certain elements accessible. IBM has taken the lead on extending MSAA, using the new IAccessible2 interface. IAccessible2 resolves the major holes left with MSAA by enhancing it, not replacing it. Switching from MSAA to other standards, such as UIA, is possible in the future, but is likely to be a very expensive endeavor. IAccessible2 allows you to keep existing MSAA support but then allows you to enhance in areas that MSAA falls short. GW Micro has been working with IBM to fully integrate IAccessible2 into its screen reader, Window-Eyes. Having Window-Eyes support IAccessible2 allows application developers to test using serious accessibility tools. IAccessible2 completes MSAA."
-- Doug Geoffray, Vice President of Development, GW Micro
The effort to enable IA2 supporting applications on other platforms is significantly smaller than with existing APIs. We have heavily designed and tested IAccessible2 to work on an entire office suite and with input from the Mozilla foundation who would implement this on Firefox. As indicated here, and Peter Korn's web log, IAccessible2 is not new in that it is derived from working APIs we helped design and implement on UNIX and Java. Also, IA2 was designed with the help of ATVs like Freedom Scientific and GW Micro during its implementation on IBM's Productivity Editors for Notes 8.
During the design of IA2 we added support for W3C ARIA unlike UIA. We believe changes to UIA are required to support Web 2.0. Going forward, IAccessible2 is an open standard and will allow others in industry to enhance it as new technology warrants. Proprietary APIs are much more restrictive and this kind of participation is very difficult. We believe they can both exist on the same platform as both are built on top of the COM transport layer.
An important benefit, for ATVs, is that they can continue to run in-process to get the performance enhancements they have realized with MSAA. This may prove difficult with the provider/client side architecture in UIA. For the non-techies running "in-process" is like talking to the person next to you vs. shouting across a street. The noisy interruptions will cause the conversation to take a lot longer to complete.
In short, because UIA has not been used by major application vendors and ATVs we believe it will take some time to be fully realized. In the mean time IAccessible2 will already have been implemented to support large rich desktop applications and W3C ARIA with full ATV support. UIA's immediate benefit may be for managed applications.
Beyond the planned IA2 support in Firefox 3 and our productivity editors in Notes 8 (Open Document Format support), we will be adding support in Eclipse. That work has just kicked off. By implementing it in Eclipse any application built using Eclipse will be able to make use of IAccessible2 and more usable access. This will allow us improve the usability of things like of rich model-based authoring tools for persons with disabilities in our Rational product line. Many vendor products work on Eclipse. IA2 will make the already strong accessibility of Eclipse more usable by persons with disabilities on Windows and once implemented it will be easier to make Eclipse fully accessible on other platforms.
I should point out that there are other vendors planning to support IA2 and I hope they will be making those announcements in the near future.
It is our hope that Microsoft will join the Free Standards Group and participate in IAccessible2. The effort dramatically improves the interoperability problem, between ATVs and application vendors, on Windows which does nothing but help Microsoft. Also, due to IAccessible2's synergy with MSAA, Java, UNIX, and UNO accessibility the uptake should be swift. Microsoft's experience in the area would be greatly welcomed and they could benefit from future accessibility innovations produced by the FSG. Like IBM, I believe Microsoft would like to move on to other accessibility challenges which have not garnered as much attention today. Here is the opportunity. I am cautiously optimistic.[Read More]