SOA meets situational applications, Part 1: Changing computing in the enterprise

As Service-Oriented Architecture (SOA) comes of age, gaining wider acceptance in the enterprise, the Web 2.0 buzz grows louder—perhaps matching the SOA hype of two years ago. Many clients and colleagues are asking, "What's the relationship between the SOA and situational applications?" "Are they orthogonal or complementary?" "Who cares about the distinction, and are these relevant to two totally different audiences?" This article covers the applicability of Web-based situational applications (SAs) to the enterprise, their relationship to SOA, and how you can use them to improve the current state of corporate IT. Learn about the IBM® experience in building the Situational Applications Environment (SAE), which was developed to support the community-based computing that takes advantage of both traditional SOA and emerging Web 2.0 technologies and approaches. Also, examine several SAs, and learn about their business situations and challenges, architectures, tangible business results, technologies that enabled the solution, and the lessons learned.

Share:

Luba Cherbakov (lubacher@us.ibm.com), IBM Distinguished Engineer, Member of the IBM Academy of Technology, IBM CIO Office, IBM

Luba Cherbakov photoLuba Cherbakov, an IBM Distinguished Engineer, is a technical leader in Application Innovation Services, IBM Global Services. She is a recognized expert in the architecture, design, and development of complex enterprise applications, using service-oriented and component-based methods. She is also an author, an advocate, and a contributor to the Service-Oriented Modeling and Architecture (SOMA) method (see Resources), reference architecture assets, the Architectural Description Standard, and grid computing offerings. A member of IEEE Computer Society and ACM, Ms. Cherbakov has a Master's degree in computer science from George Washington University, with a major in software and systems and a minor in artificial intelligence and simulation.



Andrew J. F. Bravery (andrewjf_bravery@uk.ibm.com), Senior IT Architect, IBM CIO Office, IBM

Andy Bravery photoMr. Bravery is a senior IT architect in IBM Software Group Architecture Board Incubator Projects team. He's on a rotational assignment with the IBM CIO Office as the chief architect of the Situational Applications Environment, for which he recently received an IBM Outstanding Technical Achievement award. Mr Bravery has a rich background of experience working in the emerging technology field in areas such as object-oriented programming, content management and portals, and multichannel architectures. His recent work has been around simplified programming models and Web 2.0 technologies, and how they can be applied to development in the enterprise. Mr. Bravery is a member of the British Computer Society and holds an honours degree in physics from the University of Birmingham, UK.



Aroop Pandya (apandya@us.ibm.com), Senior Software Architect, IBM CIO Office, IBM

Aroop Pandya photoAroop Pandya is a senior software architect in the office of the IBM CIO. He has extensive experience in software development with IBM WebSphere® Application Server, IBM DB2®, NNTP, and HTTPServer.



23 August 2007

Also available in Chinese

Introduction

Over the last few years, SOA has found increasingly broad acceptance and emerged as the dominant enterprise architecture to support business transformation by providing tighter linkage between business processes and enabling IT. Much has been written both about the successful SOA implementations and about the obstacles to the adoption and successful realization of SOA.

While enterprises were defining and undertaking their SOA initiatives, the recent rise of grassroots computing among both professional programmers and knowledge workers highlighted a different development approach. In this approach, those who best understand the business problem at hand develop rapid solutions without the overhead and formality of traditional IT methods. The new breed of situational applications (SAs), often developed by amateur programmers in an iterative and collaborative way, shortens the traditional edit-compile-test-run development life cycle. SAs have the potential to solve immediate business challenges in a cost-effective way, capturing the part of IT that directly impacts end users and addressing the areas that were previously unaffordable or of lower priority.

This series of articles about SAs covers the following topics:

Part 1:

  • Learn about usage patterns and technologies that contributed to the current rise of community-driven SAs development in the enterprise, and compare and contrast it with more traditional, corporate-focused SOA initiatives.
  • Examine life cycles, technologies, and social aspects of each.
  • Discover how you can use SAs in an enterprise to improve the state of corporate computing.

Part 2:

  • Learn about the IBM SAE, which was built to let individuals and small teams create ad hoc composite applications to address their immediate business needs.
  • Explore enterprise changes required to build such an environment and the challenges you must address while building it, including access to enterprise data, and privacy and security concerns.
  • Check out some lightweight development tools.

Part 3:

  • Analyze several SAs selected to represent a wide variety of business challenges that SAs are leveraged to solve. Some of these commonly occur across lines of business (LOBs) and are of general interest. Others are LOB specific, yet with typical challenges that can be successfully addressed by an SA.
  • Learn about obstacles the authors had to overcome for each of these applications, and get an architectural overview of the resulting solution and the enabling technologies, tools, and techniques used.
  • Understand the tangible business results achieved, best practices, and lessons learned with each of the SAs.

The rise of Web-based situational applications

Clay Shirky's essay titled "Situated Software" (see Resources for a link) describes a type of software that "is designed for use by a specific social group, rather than for a generic set of 'users.'" He argues that "most software built for large numbers of users or designed to last indefinitely fails at both goals anyway."

The loosely accepted term situational applications describes applications built to address a particular situation, problem, or challenge. The development life cycle of these types of applications is quite different from the traditional IT-developed, SOA-based solution. SAs are usually built by casual programmers using short, iterative development life cycles that often are measured in days or weeks, not months or years. As the requirements of a small team using the application change, the SA often continues to evolve to accommodate these changes. Significant changes in requirements may lead to an abandonment of the used application altogether; in some cases it's just easier to develop a new one than to update the one in use.

The idea of end-user computing in the enterprise is not new. Development of applications by amateur programmers using IBM Lotus® Notes®, Microsoft® Excel spreadsheets in conjunction with Microsoft Access, or other tools is widespread. What's new in this mix is the impressive growth of community-based computing coupled with an overall increase in computer skills, the introduction of new technologies, and an increased need for business agility.

The emergence of Asynchronous JavaScript + XML (Ajax)—which leverages easy access to Web-based data and rich user interface (UI) controls—combined with the Representational State Transfer (REST) architectural style of Web services offers an accessible palette for the assembly of highly interactive browser-based applications.

Similar to SOA-based solutions, SAs are seldom developed from scratch, but rather assembled from existing building blocks (or consumables as we call them). Mashups are a (better known) subset of SAs. The abundance of simple application programming interfaces (APIs) and the enablement of Ajax-style Web components (for example, Yahoo Developer Network design patterns and APIs listed on ProgrammableWeb.com) have contributed to this development style's increased popularity. Even older Web technologies, such as JavaScript, have been reinvigorated as a result.

Factors contributing to the growth in popularity of Web-based SAs among knowledge workers and professional developers are summarized in Table 1.

Table 1. Contributing factors to the rise in popularity of Web-based SAs
FactorHow it contributes to the rise of Web-based SAs
Introduction of technologies and techniques, such as Ajax, REST, Atom, and RSS feedsUsers with a lower skill base can rapidly create highly interactive applications and access data in a simple, consumable format.
Overall computer literacyA larger population of users can "program."
SOA adoptionUsers are more comfortable creating and using solutions from consumables created by others. The componentization of legacy application functionality into consumable services made it available to SAs.
Acceptance of community-based computing and collaborationDevelopers and users readily share consumables and improve on each others' work.
Proliferation of APIs and Web componentsSA builders have a richer palette for selecting consumables.
Availability of numerous mashups and SAs in the public domainNovice SA builders learn by viewing applications written by others.
Lower infrastructure costsSAs don't need to be centrally deployed; you can use individual computers as servers.
Increased requirements for business agilityLOBs can't wait for long IT development and budget cycles; they need immediate solutions to their business challenges.

Comparing SOA and SAs

Some people comment that the SOA-SA relationship is contrived. They observe that SOA is focused on providing robust enterprise-scale solutions, while SAs are do-it-yourself applications that can't scale up to address enterprise business needs.

Others see synergy between the two, as in one blog posting that describes Web 2.0 (which SAs are one aspect of) as "actually the most massive instance possible of service-oriented architecture" (see Resources for a link to this blog posting).

Our observations show that while SOA and SA have contrasting development life cycles and motivations, different usage patterns, and even dissimilar enabling technologies, they also have important aspects in common:

  • Separation and exposure of legacy applications' functionality into reusable services
  • Software as a service
  • Composition or assembly of a new solution from distributed reusable and remixable parts

Whilst with SOA the focus of compositions is on the back end, many SAs are using the Web as a platform, often focused on composing at the consumer architectural layer, or on the glass.


Development life cycle and usage patterns

Unlike the SOA-based development that often takes many months and even years, developers of SAs expect benefits of improved productivity and functionality, and a much shorter time getting from needs identification to productive application use (time-to-value). These solutions can solve some immediate business challenges in a cost-effective way—capturing the part of IT that directly impacts knowledge workers—and addresses the areas that were previously unaffordable or lower priority for inclusion into a corporate SOA project, sometimes called The Long Tail, a term first coined and popularized by Chris Anderson (see Resources for a link). By embracing this development paradigm, IT can enable the automation of business areas that were considered too niche before.

In contrast to corporate SOA development, SAs are seen as moving targets and in a state of perpetual beta. Requirements are often suggested and even implemented by users who were not the initial authors or users of the application, demonstrating a community interest and ownership mentality that is far removed from the traditional controlled environment of the corporate application. See Table 2 for a side-by-side comparison of SOA and SAs.

Table 2. SOA and SA development life cycles
SOA/Traditional IT developmentSituational applications
Time-to-value (life cycle length)Many months or yearsDays or weeks
Development phasesWell defined, following agreed-to schedule (although with frequent schedule overruns)No defined phases, milestones, or schedules

Focus on a good-enough solution to address an immediate need
Functional requirements Defined by limited number of users

IT needs to "freeze" requirements to move to development

Requirement creep often caused by changing business needs
As requirements change, SA usually changes to accommodate business changes

SA encourages unintended uses
Nonfunctional requirementsResources allocated to address concerns for performance, availability, and security

Focus on these and other nonfunctional requirements (such as scalability and maintainability) often results in robust solutions
Little or no focus on scalability, maintainability, availability, and so on
TestingBy IT with some user involvementBy users through actual use
FundingOften coincides with annual IT planning

Requires approved budgets
No formal budget

Developed and run under the radar of corporate IT

Social aspect

The deep-seated difference between SOA and SAs is in the social aspect, a principle widely considered to be at the heart of the Web 2.0 movement (from which centrally driven SOA can also benefit). Situational applications evolve through feedback from user communities; their very existence relies on this architecture of participation. The social aspects are compared in Table 3.

Table 3. Social aspect -- differences between SOA and SAs
SOASituational applications
StakeholdersLOBs' executives/corporate ITIndividual developer/self-organizing small team; user community that's formed around the SA
Targeted usersLarge, generic groupA known individual (often the application builder) or a small team; attracts unintended users
GovernanceCentralized, formalGrassroots, community based
EvolutionTop-down controlled, centrally driven, depends on available fundingOrganic, based on user feedback and participation

SA builders report improved satisfaction with their jobs and the sense of "being in control." IBM market research shows that users of SAs view them as of core importance. More than half of users regard them as "mission critical" and rate them "very important" to the success of their everyday activities, their departments, and their companies (from "Changing the corporate IT development model: Tapping the power of grassroots computing," October 2007; see Resources for a link).


Technologies

Now let's take a look at the technology changes (summarized in Table 4) that boosted the rise of SAs.

SAs' focus on user experience and highly interactive client applications readily reflects in its use of technologies and tools. Ajax, through its utilization of rich UI controls and access to Web-based data, enables major improvements to Web application responsiveness and performance. Ajax achieves high interactivity through only partial page retrieval from a server and refresh in a browser.

Table 4. SOA and SA technology preferences
SOASituational applications
MaturityHigh, proven technologiesFrequent use of less mature and emerging technologies
Preferred technologies, architectural models, and standardsWeb Services Description Language (WSDL), WS-*, SOAP, Business Process Execution Language (BPEL), Java™ 2 Platform, Enterprise Edition (J2EE), XMLAjax, REST, Atom, RSS, JavaScript Object Notation (JSON), XML, microformats, Flash, Ruby on Rails, PHP, JavaScript
User interfaceNo focusEmphasis on Rich Internet Applications (RIA) through use of technologies, such as Ajax, Adobe Flex, Adobe Integrated Runtime (AIR), OpenLaszlo, and XAML Browser Application (XBAP), as well as integration at the glass patterns
ToolsStand-alone integrated development environment (IDE)Browser based

The REST architectural style, which uses Uniform Resource Identifiers (URIs) to identify Web resources, allows equal accessibility to those resources from browsers, mobile devices, server applications, linking from e-mails, and bookmarks, which makes this programming style attractive to a wide range of client applications. Because REST style services can use the Web infrastructure for caching, they can achieve quicker response times. Given that REST doesn't need to maintain communication state, you can improve the server scalability with its use—you can use different servers to handle initial and subsequent requests.


SA applicability to the enterprise

Web 2.0 communities formed in the enterprise often see corporate IT responsible for the SOA implementation as inefficient and unable to meet business needs. While many SA enthusiasts see SOA as a heavy weight, there are some striking similarities between the two (as indicated earlier in this article).

Development of both SOA and SAs are often motivated by bringing about business flexibility. Corporate IT shouldn't ignore the complementary nature of SOA and SAs; SAs make important contributions that SOA can capitalize on, including:

  • User focus and participation.
  • Techniques for building highly interactive applications.
  • The opportunity to address areas that were considered too niche before.

Benefits of SAs

By taking advantage of the enormous potential of community-based development in the enterprise, SAs can make the benefits of an SOA-enabled organization more pervasive. The opportunities are tremendous, as summarized below.

SAs empower businesses by:

  • Encouraging innovation at departmental and individual levels—the knowledge worker can react quickly to the business situation.
  • Eliminating frustrating miscommunication and interpretation of requirements between business and IT.
  • Improving employee morale through better solutions and empowerment.

SAs improve business solutions by:

  • Creating applications that are better fit to LOB problems.
  • Satisfying short-term knowledge workers' needs.
  • Addressing niche market (The Long Tail) requirements.
  • Combining tactical SA-based solutions with the overall IT portfolio.
  • Focusing on the end user and creating a rich user experience.
  • Enriching IT data portfolio with unique, hard-to-recreate data that's created by individuals and small teams; they get richer and "cleansed" as more people use them.

SAs help improve ROI by:

  • Shortening the development life cycle.
  • Eliminating formal development activities when they're not needed, thus increasing productivity.
  • Cutting the costs of the overall development process.
  • Creating applications that weren't written before, because they weren't affordable.

Challenges of SAs

At the same time, introducing SAs into the corporate computing environment increases challenges for the IT department, as summarized below.

IT department can view SAs as "controlled" chaos because:

  • There's no formal budget; SAs are built and run under the radar of central IT.
  • The architecture of participation includes users as co-developers.
  • There's spontaneous development and evolution with applications being in perpetual beta.
  • There's little focus on nonfunctional requirements, such as scalability, availability, and maintainability.
  • Best practices of adoption in enterprises don't exist.

Integration of SAs is pushed to the edge because:

  • Control is moving from applications to fine-grained services and data feeds.
  • Internal and external services and data feeds are mashed inside the enterprise firewalls.
  • Applications are shared on a global scale.
  • Businesses must embrace the Web as a platform on which to conduct business.

Management of situational and enterprise applications is complicated due to:

  • Multiple development and operations environments, such as J2EE; Linux, Apache, MySQL, PHP/Perl (LAMP); and Microsoft .NET.
  • Increasingly harder end-to-end systems management in a mixed environment (for example, monitoring, event analysis, root-cause detection, and patch management).

SAs create data security, privacy, and integrity challenges, including:

  • Enterprise data as an enabler; opening up enterprise data sources is a long and complicated process, involving many stakeholders.
  • The ownership of unique data produced by LOBs and individuals raising data-providence questions.
  • Encouraging unintended uses of SAs (formal control versus education and guidance).

Summary

Enabling SAs in the enterprise requires additional resiliency from the existing corporate infrastructure. SOA can help mitigate some of the challenges by, for example, reusing services and infrastructure created by SOA initiatives for SA management, operation, and governance.

While many IT departments are skeptical about the usefulness of SAs to corporate computing or wary of using externally developed APIs inside their firewalls, the IBM CIO office decided to unleash the use of SAs inside the enterprise. To better understand both the challenges and opportunities of adopting community-based development, the IBM CIO office established an initiative called The Situational Applications Environment (SAE). Envisioned as a living lab experiment to observe and harvest best practices, the SAE is depicted in Figure 1.

Figure 1. The SAE concept inside IBM
SAE

Stay tuned for Part 2 of this article series, which describes the architecture and lessons learned from building this SAE. Here's a summary of early business benefits already observed:

  • Collaborating communities established themselves around common business topics of interest, thus increasing reuse of assets and solutions.
  • Some applications were evolved to include additional and improved features not initially envisioned by application builders. Through their comments and direct contributions, users made applications more useful to themselves and others.
  • Business users' needs forced the IT department to open corporate data sources as consumable services that enabled fine-grained access to data.
  • Situational development allowed LOBs to take more control in deciding which projects get done and how they are done—either with SAs or traditional development.

Leveraging SA development with SOA in enterprise computing has the potential to fundamentally transform IT's role from the solutions developer to the solutions enabler. We believe this is a required change for IT to remain relevant.

Resources

Learn

Get products and technologies

  • Innovate your next development project with IBM trial software, available for download or on DVD.

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into SOA and web services on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=SOA and web services
ArticleID=251188
ArticleTitle=SOA meets situational applications, Part 1: Changing computing in the enterprise
publish-date=08232007