The first installment of this series about situational applications described both usage patterns and technologies that contributed to the current rise of community-driven SA development in the enterprise, and compared and contrasted it with the more traditional, corporate-focused SOA initiatives. The article examined life cycles, technologies, and social aspects of each. Then it described how you can use SAs in an enterprise to improve the state of corporate computing.
This second article in the series describes the IBM SAE that was built to help individuals and small teams create ad hoc composite applications to address their immediate business needs. Here you learn about the changes in the enterprise that are required to build such an environment and the challenges you must address while building it. In particular, you examine access to enterprise data, the life cycle of SAs, and their influence on traditional application development. You also learn about some lightweight development tools.
The final article of the series will examine several SAs and describe their business situation and challenges, architecture, tangible business results, the technologies that enabled each solution, and lessons learned.
The recent emergence of mashup and API hubs, such as Programmableweb.com (see Resources for a link), has revealed a significant buzz developing around the ad hoc creation of simple applications to meet immediate and highly personalized needs. The SAE was built to examine whether such an ecosystem could be built within an enterprise, offering more autonomy to the lines of business (LOBs) by shifting some just-in-time business automation responsibilities from corporate IT to small teams and individuals.
To stimulate a vibrant SA enterprise ecosystem, it's important to bring together users with a range of complementary skills.
Business-focused users know where business value can be quickly realized through the targeted creation of new applications. Technology-focused users understand programming concepts and data complexities. The distinct SA community roles are summarized in Table 1.
Table 1. User roles in an SA-focused community
|SA users||Business user: |
|SA assemblers|| Business power user or LOB developer: |
|Consumable builders|| LOB or IT developer: |
The emergence of Web 2.0 social interactions means that the immense power and creativity of this corporate workforce is more accessible and connected than ever before. The SAE provides the facilities required to sustain such interaction between these user groups, providing a "living" lab where business benefits of SAs in the enterprise can be observed.
The central focus for the environment is the SAE home page. It draws the attention of users to the latest and most popular additions from the community (such as SAs, consumables, and the latest forum threads), introduces tooling in a task-driven manner, and raises awareness of corporate guidelines for interpreting data and use of internal and external services.
Figure 1 highlights some of the key features of the SAE home page.
Figure 1. The SAE home page
Figure 2 lays out the key SAE architectural components.
Figure 2. The SAE architecture components
Each catalog entry describes the asset's details and provides examples of how to use it and whom to contact about it. The details include minimal categorization augmented with tagging. Users can comment on the entry and register details of their own usage examples. They can also add tags, and both the entry and tags can be rated based on popularity and relevance. This user-generated taxonomy, or folksonomy as it is becoming known, is then used to filter entries along with more traditional keyword search techniques.
While the catalog is open for anonymous browsing, contributions require a user's identity confirmation through the corporate authentication service. This identity confirmation associates the contributor's contact details from a corporate directory with the catalog entry, thus leading to community building.
The assets themselves may be hosted on the SAE infrastructure, on servers owned by the asset contributors, or by external partners.
The SAE provides both keyword and tag-based search services accessible to other applications through Atom data feeds and OpenSearch APIs. This means that the SAE is both a mashup itself and provides consumables that can be included in other mashups.
Most catalog views have associated Atom feeds, which can be subscribed to through feed readers, enabling users to get new information or community discussion updates pushed to them rather than having to visit the catalog.
SAE tools and utilities
The SAE provides tools that enable the acquisition of data that may only be available on existing HTML Web pages. Again, the focus of these tools is to create REST service APIs in front of the scraping logic so that the consumer of the service is unaware of what's happening behind the scenes.
The SAE also includes an instance of the IBM Mashup Starter Kit, which is comprised of QEDWiki (for page composition from widgets), Mashup Hub (for cataloging widgets and data feeds), and a data mashup tool that allows you to create new data feeds by combining existing enterprise data, similar to Yahoo! Pipes (see Resources for a link).
Project Zero (see Resources for a link) introduces a simple environment for creating, assembling, and executing applications based on popular Web technologies. Its programming models are built around REST, Atom, and Ajax patterns and include a scripting run time for Groovy and PHP.
Table 2 summarizes the capabilities of selected SAE tools.
Table 2. Summary of SAE tooling capabilities
After a user builds a consumable or an SA, he or she has to deploy (host) it somewhere. Some tools described above, mashup makers as they are becoming known, remove the deployment step by combining build and execute environments. Users build Web pages from a palette of components, test, and then share the resulting application without infrastructure considerations. The host platform manages the complete cycle of assemble, run, and share, exploiting rich client-side components to give the user an integrated development experience. Other tools, used in an offline development mode similar to a traditional integrated development environment (IDE), require hosting.
The SAE provides a set of lightweight virtual hosting environments that remove SA builders' responsibility for the infrastructure management. These hosted stacks are chosen to match those commonly used for SA building and are set up to be self-serviced to minimize central administration overhead. A typical hosted stack includes a Web server, server-side scripting capability, and data storage. Users can upload code and make small configuration changes that affect only their virtual host. Currently, basic Linux, Apache, MySQL, PHP/Perl (LAMP) and Ruby are supported in the SAE. Future plans include adding a similar offering for Project Zero applications in addition to facilities like generic data storage, plug-in download libraries, and data mediation services.
Observations of an ecosystem in action
Since the SAE launch in December 2006, growing community activities have brought some interesting insight into how such an environment can provide value to the enterprise.
Application life cycle
Whilst many SAs live for only a short period before being abandoned, or never become widely used beyond the original team that created them, some capture the imagination of a large number of people.
IBM Travel Maps, one of the first SAs registered in the SAE, had originally been developed by two researchers as the winning entry in a PHP contest. Travel Maps mashed information useful to travelers, such as IBM site locations, recommended hotels, airport, rental car, restaurant locations, and other points of interest on a navigable map.
Although an IBM Business Partner provides the IBM Online Travel Reservations (OTR) self-service system for planning and booking corporate travel, the OTR currently doesn't include convenient location-mapping features or information about local points of interest. Travel Maps offers these missing features. The application became so popular that the IBM Real Estate Operations management group added it to the IBM intranet travel site only three weeks after the SAE launch.
The evolution of the application has continued through the architecture of participation, a Web 2.0 trait used to describe community participation. Newer travel mashups added weather, flight status, and client-related information.
Many IBM Travel Maps-related community comments were about the data quality within the application. Much of this data had been hidden from end users in the past, perhaps only used by a small number of people who weren't intimately connected to this information. After this information was exposed to those who do have the intimate knowledge of it, inaccuracies were quickly pointed out. Geographic details of IBM locations and other local points of interest were recognized by local employees as appearing in the wrong places on the map, and they reported the problems through the SAE. The details of the inaccuracies were quickly pushed to the data owners who could alter the original data.
This community moderation of data is similar to participation seen in popular Web 2.0 Web sites, such as Wikipedia. Work is now going on in the SAE to automate the end-user feedback collection and support a level of community moderation compliant with enterprise data governance and security policies.
SAs are built to be just good enough to get a particular job done. It's often not obvious to users that an application is anything other than a production-ready tool, especially to those outside the original team of interest who built it. If business decisions are to be made based on the usage of an SA, it's important that prospective users understand the SA builder's original objectives.
Best practice in the SAE recommends that SA builders provide sufficient detail in the description of their SA and include links back from the SA to the SAE catalog so users can discover the provenance of the SA, rate it, and provide feedback. A special widget has been created that builders can include in their SA to provide such a link. In future releases, this widget will be integrated with the SA builder tooling so that it becomes an integral part of any SA built with that tool.
In traditional application development, extensive resources are used to ensure good performance by building well-designed end-to-end architectures. With SAs, an application is built quickly, often with components that aren't under the developer's control. For example, the only way to acquire data might be through an inefficient or complex query that spans several data sources, including Web services, multiple databases, and, through scraping techniques, Web pages. There are few opportunities to optimize data gathering in this scenario.
This is a trade off between making data available, where it was not in the past, and making it perform well. Caching techniques can help with this, especially as SAs are often built around relatively nonvolatile data.
Much community comment about SAs complains that the performance is poor—sadly the end user rarely appreciates the underlying complexity of the SA and therefore has unreasonable expectations of it. Of course, when an SA becomes popular and key to the business, the corporate IT needs to consider whether to invest in turning the SA, or at least elements of it, into a more robust performing application.
For any business, the value a technology delivers is the key success indicator. The SAE initiative has worked with LOBs to identify SAs that could demonstrate measurable business value. Good candidate SAs focus on automating highly manual data manipulation tasks.
In one such case, a 50% increase in productivity was observed across a team of 25 analysts using a new SA; in another, the time taken to complete a task was reduced from 2.5 hours to four minutes! Details of these and other SAs are examined in the third article of this series.
The provision of an ecosystem to support development and usage of SAs in an enterprise is crucial if the business benefits of such applications are to be realized. CIOs have an important and challenging role to change the enterprise culture to one that encourages and embraces innovative thinking and individual self sufficiency. IBM is actively removing existing technological and cultural barriers to support an entrepreneurial atmosphere. SAE was designed as an evolving experiment in enterprise-wide enablement and adoption of situational applications.
An underlying SOA is required to provide a sufficient number of consumables to grow this environment and give it the momentum needed to become self sustaining. Many organizations are adopting SOA as a strategy, but opening up services to SA builders is essential if corporate IT is to act as an enabler to the growth of SAs as a key tool.
Although the initial SAE release addressed some of the challenges described here, there's much left to explore and discover, such as:
- Access to enterprise data sources through automated role-based entitlement systems.
- Data provenance.
- Strengthening overall governance, especially in the area of externally produced consumables.
- The evolution of mashup makers targeting business users.
Better tools for building SAs and consumables are needed to enforce corporate security and governance policies without placing a burden on the SA builder.
The SAE is being extended with facilities for submission of details of business needs to allow business users to articulate their problems to the technical community that can help solve them by building SAs. This wish-list function is enabled with the same Web 2.0-style collaboration functions as other parts of the SAE catalog.
In Part 3 of this article series, you examine several situational applications. These applications have been selected to represent a wide variety of business challenges that SAs are leveraged to solve. Some of these are commonly occurring across lines of business and are of general interest. Others are LOB specific, yet with typical challenges that can be successfully addressed by an SA.
In addition to the overall business context, for each examined application we will describe obstacles we had to overcome in creating them and an architectural overview of the resulting solution. You'll also look at enabling technologies, tools, and techniques used and learn about tangible business results achieved by each, best practices, and lessons learned.
- Check out the mashup and API hub Programmableweb.com.
- Visit the Yahoo! Pipes site.
- Check out Project Zero.
- "Changing the corporate IT development model: Tapping the power of grassroots computing" provides further examination into how situational apps are used.
- Dion Hinchcliffe from ZDNet offers his views on the impact of grassroots computing in "Mashups: The next major new software development model?".
- "Representational State Transfer (REST)" describes the REST architectural style for distributed systems.
- The SOA and Web services zone on IBM developerWorks hosts hundreds of informative articles and introductory, intermediate, and advanced tutorials on how to develop Web services applications.
- Play in the IBM SOA Sandbox! Increase your SOA skills through practical, hands-on experience with the IBM SOA entry points.
- The IBM SOA Web site offers an overview of SOA and how IBM can help you get there.
- Stay current with developerWorks technical events and webcasts. Check out the following SOA and Web services tech briefings in particular:
- Browse for books on these and other technical topics at the Safari bookstore.
- Check out a quick Web services on demand demo.
- Get an RSS feed for this series. (Find out more about RSS.)
Get products and technologies
- Innovate your next development project with IBM trial software, available for download or on DVD.
- Participate in the discussion forum.
- Get involved in the developerWorks community by participating in developerWorks blogs.