SOA meets situational applications, Part 2: Building the IBM Situational Applications Environment

The first article of this series explained the applicability of Web-based situational applications (SAs) to the enterprise, their relationship to Service-Oriented Architecture (SOA), and how they can be used to improve the current state of corporate IT. This article describes the IBM® experience in building the Situational Applications Environment (SAE), which has been developed to support the community-based computing that takes advantage of both traditional SOA and emerging Web 2.0 technologies and approaches.

Andrew J. F. Bravery (andrewjf_bravery@uk.ibm.com), Senior IT Architect, IBM CIO Office, IBM

Andy Bravery photoMr. Bravery is a senior IT architect in IBM Software Group Architecture Board Incubator Projects team. He's on a rotational assignment with the IBM CIO Office as the chief architect of the Situational Applications Environment, for which he recently received an IBM Outstanding Technical Achievement award. Mr Bravery has a rich background of experience working in the emerging technology field in areas such as object-oriented programming, content management and portals, and multichannel architectures. His recent work has been around simplified programming models and Web 2.0 technologies, and how they can be applied to development in the enterprise. Mr. Bravery is a member of the British Computer Society and holds an honours degree in physics from the University of Birmingham, UK.



Luba Cherbakov (lubacher@us.ibm.com), IBM Distinguished Engineer, Member of the IBM Academy of Technology, IBM CIO Office, IBM

Luba Cherbakov photoMs. Cherbakov, an IBM Distinguished Engineer and a member of the IBM Academy of Technology, is a key technical leader in the IBM CIO Office. Currently she drives the Situational Applications Environment Initiative that brings together Web 2.0 technologies, growing availability of services, and enterprise data. Ms. Cherbakov is a recognized expert in enterprise architecture design and development of complex distributed applications using service-oriented and component-based methods. She's an author and contributor to the IBM Service-Oriented Modeling and Architecture (SOMA) method, reference architectures, the Architectural Description Standard, and grid computing assets. Ms. Cherbakov is a two-time recipient of the IBM Outstanding Technical Achievement Award and a recipient of the IBM Corporate Award, the highest technical award for unique technical contributions of superior business value. She's a member of the IEEE Computer Society, the Society of Women Engineers, and the Association for Computing Machinery, and holds a master's degree in computer science from George Washington University.



Aroop Pandya (apandya@us.ibm.com), Senior Software Architect, IBM CIO Office, IBM

Aroop Pandya photoAroop Pandya is a senior software architect in the office of the IBM CIO. He has extensive experience in software development with IBM WebSphere® Application Server, IBM DB2®, NNTP, and HTTPServer.



10 January 2008

Also available in Chinese Russian

Introduction

The first installment of this series about situational applications described both usage patterns and technologies that contributed to the current rise of community-driven SA development in the enterprise, and compared and contrasted it with the more traditional, corporate-focused SOA initiatives. The article examined life cycles, technologies, and social aspects of each. Then it described how you can use SAs in an enterprise to improve the state of corporate computing.

This second article in the series describes the IBM SAE that was built to help individuals and small teams create ad hoc composite applications to address their immediate business needs. Here you learn about the changes in the enterprise that are required to build such an environment and the challenges you must address while building it. In particular, you examine access to enterprise data, the life cycle of SAs, and their influence on traditional application development. You also learn about some lightweight development tools.

The final article of the series will examine several SAs and describe their business situation and challenges, architecture, tangible business results, the technologies that enabled each solution, and lessons learned.

The recent emergence of mashup and API hubs, such as Programmableweb.com (see Resources for a link), has revealed a significant buzz developing around the ad hoc creation of simple applications to meet immediate and highly personalized needs. The SAE was built to examine whether such an ecosystem could be built within an enterprise, offering more autonomy to the lines of business (LOBs) by shifting some just-in-time business automation responsibilities from corporate IT to small teams and individuals.

To stimulate a vibrant SA enterprise ecosystem, it's important to bring together users with a range of complementary skills.

Business-focused users know where business value can be quickly realized through the targeted creation of new applications. Technology-focused users understand programming concepts and data complexities. The distinct SA community roles are summarized in Table 1.

Table 1. User roles in an SA-focused community
RolesAttributesRequirements
SA usersBusiness user:
  • Knows the business need
  • Needs quick answers
  • Has standard desktop tools
  • Find SAs quickly and use them
  • Rate SAs and comment on them
  • Describe business needs for new SAs
  • Demand real business benefits from SAs
SA assemblers Business power user or LOB developer:
  • Understands spreadsheet formulae
  • Knows/is close to business need
  • Is capable of SA composition
  • Uses browser-based tooling
  • Can access a range of tools to fit skills and domain expertise
  • Compose applications from consumables
  • Provide guidance via programming-by-example, templates, utilities, and so on
  • Share SAs and collaborate to improve them
Consumable builders LOB or IT developer:
  • Has traditional programming skills
  • Understands integration issues
  • Uses a range of programming tools
  • Can access enterprise data
  • Use tools and utilities to build consumables
  • Use patterns to guide reusable asset creation
  • Use lightweight hosting to run consumables

The emergence of Web 2.0 social interactions means that the immense power and creativity of this corporate workforce is more accessible and connected than ever before. The SAE provides the facilities required to sustain such interaction between these user groups, providing a "living" lab where business benefits of SAs in the enterprise can be observed.


SAE architecture

The central focus for the environment is the SAE home page. It draws the attention of users to the latest and most popular additions from the community (such as SAs, consumables, and the latest forum threads), introduces tooling in a task-driven manner, and raises awareness of corporate guidelines for interpreting data and use of internal and external services.

Figure 1 highlights some of the key features of the SAE home page.

Figure 1. The SAE home page
The SAE home page

Figure 2 lays out the key SAE architectural components.

Figure 2. The SAE architecture components
The SAE architecture components

The catalog stores assets created or discovered by the community. These assets are either links to SAs or parts from which SAs are constructed. They might be Web services, JavaScript widgets, APIs, or code snippets—we call them consumables.

Each catalog entry describes the asset's details and provides examples of how to use it and whom to contact about it. The details include minimal categorization augmented with tagging. Users can comment on the entry and register details of their own usage examples. They can also add tags, and both the entry and tags can be rated based on popularity and relevance. This user-generated taxonomy, or folksonomy as it is becoming known, is then used to filter entries along with more traditional keyword search techniques.

While the catalog is open for anonymous browsing, contributions require a user's identity confirmation through the corporate authentication service. This identity confirmation associates the contributor's contact details from a corporate directory with the catalog entry, thus leading to community building.

The assets themselves may be hosted on the SAE infrastructure, on servers owned by the asset contributors, or by external partners.

The SAE provides both keyword and tag-based search services accessible to other applications through Atom data feeds and OpenSearch APIs. This means that the SAE is both a mashup itself and provides consumables that can be included in other mashups.

Most catalog views have associated Atom feeds, which can be subscribed to through feed readers, enabling users to get new information or community discussion updates pushed to them rather than having to visit the catalog.

SAE tools and utilities

SAE tools support technology patterns that are being adopted by SA builders, such as Representational State Transfer (REST)-style Web services and Asynchronous JavaScript + XML (Ajax)-enabled UI widgets. By adopting these patterns as a guideline, developers are encouraged to build consumables that are compatible with each other and the provided tool set.

The Enterprise Business Innovation (EBI) REST Framework is a tool for building REST Web services on top of data stored in the enterprise's relational databases. This self-service framework enables users with rudimentary SQL knowledge to define services and choose the response protocol, such as JavaScript Object Notation (JSON) or XML.

The SAE provides tools that enable the acquisition of data that may only be available on existing HTML Web pages. Again, the focus of these tools is to create REST service APIs in front of the scraping logic so that the consumer of the service is unaware of what's happening behind the scenes.

The SAE also includes an instance of the IBM Mashup Starter Kit, which is comprised of QEDWiki (for page composition from widgets), Mashup Hub (for cataloging widgets and data feeds), and a data mashup tool that allows you to create new data feeds by combining existing enterprise data, similar to Yahoo! Pipes (see Resources for a link).

Project Zero (see Resources for a link) introduces a simple environment for creating, assembling, and executing applications based on popular Web technologies. Its programming models are built around REST, Atom, and Ajax patterns and include a scripting run time for Groovy and PHP.

Table 2 summarizes the capabilities of selected SAE tools.

Table 2. Summary of SAE tooling capabilities
Situational applicationsConsumables
QEDWiki
  • Page layout tool
  • Extensible widget library
  • Consumes RSS and Atom feeds
  • Widget attribute binding
Project Zero
  • Dojo support
  • Automated page creation
  • Eclipse tooling
EBI framework
  • Produces REST Web services
  • Extensible widget library
  • Consumes RSS and Atom feeds
  • Widget attribute binding
Mashup Hub
  • Produces RSS and Atom feeds
  • Enterprise data access
  • Rich data operator library
  • Visual composition tool
Project Zero
  • Produces and consumes REST services
  • Produces and consumes Atom feeds
  • Scripting with PHP or Groovy

SAE hosting

After a user builds a consumable or an SA, he or she has to deploy (host) it somewhere. Some tools described above, mashup makers as they are becoming known, remove the deployment step by combining build and execute environments. Users build Web pages from a palette of components, test, and then share the resulting application without infrastructure considerations. The host platform manages the complete cycle of assemble, run, and share, exploiting rich client-side components to give the user an integrated development experience. Other tools, used in an offline development mode similar to a traditional integrated development environment (IDE), require hosting.

The SAE provides a set of lightweight virtual hosting environments that remove SA builders' responsibility for the infrastructure management. These hosted stacks are chosen to match those commonly used for SA building and are set up to be self-serviced to minimize central administration overhead. A typical hosted stack includes a Web server, server-side scripting capability, and data storage. Users can upload code and make small configuration changes that affect only their virtual host. Currently, basic Linux, Apache, MySQL, PHP/Perl (LAMP) and Ruby are supported in the SAE. Future plans include adding a similar offering for Project Zero applications in addition to facilities like generic data storage, plug-in download libraries, and data mediation services.


Observations of an ecosystem in action

Since the SAE launch in December 2006, growing community activities have brought some interesting insight into how such an environment can provide value to the enterprise.

Application life cycle

Whilst many SAs live for only a short period before being abandoned, or never become widely used beyond the original team that created them, some capture the imagination of a large number of people.

IBM Travel Maps, one of the first SAs registered in the SAE, had originally been developed by two researchers as the winning entry in a PHP contest. Travel Maps mashed information useful to travelers, such as IBM site locations, recommended hotels, airport, rental car, restaurant locations, and other points of interest on a navigable map.

Although an IBM Business Partner provides the IBM Online Travel Reservations (OTR) self-service system for planning and booking corporate travel, the OTR currently doesn't include convenient location-mapping features or information about local points of interest. Travel Maps offers these missing features. The application became so popular that the IBM Real Estate Operations management group added it to the IBM intranet travel site only three weeks after the SAE launch.

The evolution of the application has continued through the architecture of participation, a Web 2.0 trait used to describe community participation. Newer travel mashups added weather, flight status, and client-related information.

Data moderation

Many IBM Travel Maps-related community comments were about the data quality within the application. Much of this data had been hidden from end users in the past, perhaps only used by a small number of people who weren't intimately connected to this information. After this information was exposed to those who do have the intimate knowledge of it, inaccuracies were quickly pointed out. Geographic details of IBM locations and other local points of interest were recognized by local employees as appearing in the wrong places on the map, and they reported the problems through the SAE. The details of the inaccuracies were quickly pushed to the data owners who could alter the original data.

This community moderation of data is similar to participation seen in popular Web 2.0 Web sites, such as Wikipedia. Work is now going on in the SAE to automate the end-user feedback collection and support a level of community moderation compliant with enterprise data governance and security policies.

Managing expectations

SAs are built to be just good enough to get a particular job done. It's often not obvious to users that an application is anything other than a production-ready tool, especially to those outside the original team of interest who built it. If business decisions are to be made based on the usage of an SA, it's important that prospective users understand the SA builder's original objectives.

Best practice in the SAE recommends that SA builders provide sufficient detail in the description of their SA and include links back from the SA to the SAE catalog so users can discover the provenance of the SA, rate it, and provide feedback. A special widget has been created that builders can include in their SA to provide such a link. In future releases, this widget will be integrated with the SA builder tooling so that it becomes an integral part of any SA built with that tool.

Performance

In traditional application development, extensive resources are used to ensure good performance by building well-designed end-to-end architectures. With SAs, an application is built quickly, often with components that aren't under the developer's control. For example, the only way to acquire data might be through an inefficient or complex query that spans several data sources, including Web services, multiple databases, and, through scraping techniques, Web pages. There are few opportunities to optimize data gathering in this scenario.

This is a trade off between making data available, where it was not in the past, and making it perform well. Caching techniques can help with this, especially as SAs are often built around relatively nonvolatile data.

Much community comment about SAs complains that the performance is poor—sadly the end user rarely appreciates the underlying complexity of the SA and therefore has unreasonable expectations of it. Of course, when an SA becomes popular and key to the business, the corporate IT needs to consider whether to invest in turning the SA, or at least elements of it, into a more robust performing application.

Business value

For any business, the value a technology delivers is the key success indicator. The SAE initiative has worked with LOBs to identify SAs that could demonstrate measurable business value. Good candidate SAs focus on automating highly manual data manipulation tasks.

In one such case, a 50% increase in productivity was observed across a team of 25 analysts using a new SA; in another, the time taken to complete a task was reduced from 2.5 hours to four minutes! Details of these and other SAs are examined in the third article of this series.


Future direction

The provision of an ecosystem to support development and usage of SAs in an enterprise is crucial if the business benefits of such applications are to be realized. CIOs have an important and challenging role to change the enterprise culture to one that encourages and embraces innovative thinking and individual self sufficiency. IBM is actively removing existing technological and cultural barriers to support an entrepreneurial atmosphere. SAE was designed as an evolving experiment in enterprise-wide enablement and adoption of situational applications.

An underlying SOA is required to provide a sufficient number of consumables to grow this environment and give it the momentum needed to become self sustaining. Many organizations are adopting SOA as a strategy, but opening up services to SA builders is essential if corporate IT is to act as an enabler to the growth of SAs as a key tool.

Although the initial SAE release addressed some of the challenges described here, there's much left to explore and discover, such as:

  • Access to enterprise data sources through automated role-based entitlement systems.
  • Data provenance.
  • Strengthening overall governance, especially in the area of externally produced consumables.
  • The evolution of mashup makers targeting business users.

Better tools for building SAs and consumables are needed to enforce corporate security and governance policies without placing a burden on the SA builder.

The SAE is being extended with facilities for submission of details of business needs to allow business users to articulate their problems to the technical community that can help solve them by building SAs. This wish-list function is enabled with the same Web 2.0-style collaboration functions as other parts of the SAE catalog.

In Part 3 of this article series, you examine several situational applications. These applications have been selected to represent a wide variety of business challenges that SAs are leveraged to solve. Some of these are commonly occurring across lines of business and are of general interest. Others are LOB specific, yet with typical challenges that can be successfully addressed by an SA.

In addition to the overall business context, for each examined application we will describe obstacles we had to overcome in creating them and an architectural overview of the resulting solution. You'll also look at enabling technologies, tools, and techniques used and learn about tangible business results achieved by each, best practices, and lessons learned.

Resources

Learn

Get products and technologies

  • Innovate your next development project with IBM trial software, available for download or on DVD.

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into SOA and web services on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=SOA and web services
ArticleID=281349
ArticleTitle=SOA meets situational applications, Part 2: Building the IBM Situational Applications Environment
publish-date=01102008