There has been a lot of buzz lately in tech magazines, the blogosphere, and at IT industry events about SOA and Web 2.0. Although the buzz indicates a growing interest in the technical community, most business executives don't understand how these two modes of information exchange might interoperate and what it means to them in the future. The purpose of this article is to shed light on the relationships between SOA and Web 2.0, and to describe a future Web services delivery model that incorporates IT governance.
Consumer power in contemporary culture
"The consumer is king!" "The customer is always right!" We have heard these catch phrases probably since we were all in middle school, but today, the meaning of these concepts is being expressed in new forms. Take a look at a few of the more popular TV shows in America today, and you see consumers as the "bosses." Shows like American Idol and So You Think You Can Dance are literally driven by votes from the viewing public. Trends come and go in the entertainment industry, but this degree of consumer participation represents a powerful, populist move away from buttoned-down judges to the decisions made by average TV watchers.
A 2006 film called Snakes on a Plane received mass publicity before it was released through blogs, podcasts, etc. -- all of which was driven by fans who even made demands through Web forums on what some of the movie's dialogue should be. The production unit went back and shot scenes as demanded in blogs; now that's power in the hands of consumers. Although the viral marketing for this movie happened by accident,1 the events tell us that if organizations provide consumers with the right tools and freedom to give feedback and promote the product, consumers will quickly show what they think of a product and act accordingly.
The strength of consumers' power has become the backbone of many Web-based businesses. Online collaboration portals like MySpace.com and Facebook.com rely simply on user participation and enrichment. Companies are fighting hard to find ways to satisfy information- and power-hungry consumers. To top it off, consumers are demanding all these services faster, better, and cheaper. And the worst thing is, if you can't satisfy them, you must be ready to face some strong unfiltered criticism. Increased customer demands are not limited to TV shows, movies, or online businesses who serve the masses; also included are traditional mega enterprises, where ever-increasing customer demands lead to product and process improvements. Not too long ago, one Internet Service Provider (ISP) got in trouble when a recording of a not-so-pleasant conversation between a customer and the call service center agent was posted on a blog. The customer was trying to terminate the service, and the agent was trying to keep him as a subscriber. The notoriety generated through this blog entry forced the famous ISP to change its process for customer termination so customers can more easily unsubscribe to their service.
Of course, companies have long sought to discover what the marketplace wants through direct customer communication, such as written surveys, questionnaires, and phone calls. Most of the time, data collected through these traditional means is synthesized and presented for analysis in some convenient format. But this is nothing like a customer voicing an unfiltered opinion through newer technologies that enable direct, collaborative communication. In other words, blogs, wikis, and interactive forums have created a more direct way for customers to affect a product or service than was possible in yesteryears.
Meanwhile, back at the corporate ranch...
While consumers have been busy defining or exploring new modes of entertainment and communication, businesses are exploiting some of the same technologies to survive competitive threat and increase shareholder value. There was a time when companies thought that laying off parts of an IT workforce would help increase the bottom line. Then came the idea that outsourcing to distant lands will reduce delivery costs or help the organization focus on core competencies, all of which will equate to more profit for the business. However, in the last few years, as more and more companies started outsourcing or off-shoring, the number of off-shore providers has grown exponentially, all of whom are ever-ready to capitalize on the price-sensitive customer. This increase in supply of low-cost outsource providers has made lower delivery costs a de facto standard, i.e., not much of a competitive advantage anymore.
IT organizations have figured out that operational success will depend more on optimizing business operations through increased IT effectiveness. IT's response has been to decentralize IT operations. Many IT organizations are moving to a transactional, service-oriented culture where business needs are met through delivering a congregation of services. The goal is to create many different services, which can coordinate, reuse, and re-assemble assets in order to provide the best value for the business.
In order to be effective, a few core concepts have been established for these services to be well orchestrated and deliver value.
- Standards. Well-defined APIs and standards like SOAP, Web Services Descripion Language (WSDL), etc. allow different services to work together to deliver business results. SOAP is a protocol for exchanging XML-based messages over a computer network, mainly using HTTP. SOAP is the foundation layer for the Web services stack, providing a basic messaging framework that allows for more abstract layers to build on. WSDL is an XML-based service description that shows how to communicate using the Web service. It is the protocol that binds the message format required to interact with the Web services listed in its directory. In the past, we never saw the broad and deep consensus we see today among the IT industry heavyweights regarding support for standards. Standards existed, but they didn't have the extensive buy-in that they do today.
- Customer commitment. Customers today understand the importance of working with IT organizations to make sure that their applications are built in a way that gives them maximum flexibility for the future. We see services acting as a unifying force for business, IT, and the customer. The flexibility of business processes is drawing customers along with line-of-business management to discuss operations with IT.
- Level of abstraction. There is more of a one-to-one relationship between steps in a business process and the services that support those steps. What this means is that now we have a larger service that does something like check a new customer's credit rating or initiates a shipment. This new service could be a combination of many small services. Technically speaking, the supported operations and messages are described abstractly, then bound to a concrete network protocol and message format. WSDL describes the public interface to the Web service. WSDL is often used in combination with SOAP and XML Schema to provide Web services over the Internet.
- Reusability. It is important to create services that can be reused and re-orchestrated regardless of whether they're newly created services or existing IT assets that have been converted to services.
The rise of SOA
The major way big enterprises are addressing the creation of a service-based delivery model is by using SOA, or service-oriented architecture. SOA is a type of software architecture that defines the use of services to support the requirements of the business. In an SOA environment, resources on a network are presented as independent services that can be accessed without knowledge of their underlying platform implementation. Services are defined as business processes. When thinking about business processes, we have to think beyond IT or software and think more in terms of business. We have to think about what our company does on a day-to-day basis and break it into meaningful business objects. We can then take these business objects and transform them into reusable business components (for example, by taking out hard-coded logic, including open APIs, etc.), which can be snapped into place with other components (services) to create application functionality.
The value proposition lies in SOA's multi-layered application style, which allows different organizations within the company to share data, logic, and artifacts among applications and usage models. SOA is all about agility, interoperability, and reusability. SOA is primarily based on Web services standards, but also can be implemented using any service-based technology. Loosely-coupled services connect with one another via a formal definition such as WSDL, which is independent of the underlying programming language or development platform. For example, it is very realistic for an application to consume a service built on the .Net platform while using some services written on the J2EE platform. In short, SOA makes it easy to snap together services into a business process just like snapping together building blocks into a structure.
One interesting observation about SOA is that it still encapsulates entities and services within an organizational firewall. Different services can be leveraged from partners and other trusted resources, but the openness only extends so much. For an example, in today's SOA practiced model, it is not common to see open APIs in critical applications allowing developers from outside the organization to come and add onto the composite application as they wish. The roles of this secured environment is very important for bigger organizations who are dealing with sensitive customer data and are very concerned about system downtime. In summary, it can be said the enterprise composite applications have found their answer in SOA to produce a reusable, interoperable, and safe process to share services and artifacts.
A brief tour of Web 2.0
The term "Web 2.0" was originally coined by technologists from Tim O'Reilly's organization to describe the next generation of Web-based application development platforms. Web 2.0 is not about one particular technology or a description of a set of tools -- it is more about leveraging the power of the World Wide Web to build composite applications in a service-oriented fashion. Conceptually, Web 2.0 applications integrate data and services coming from multiple locations to create a resulting service that is presented to its users. This newly created service can also be fed as input to other Web-based services. This type of on-demand delivery model feeds the appetite of today's information-hungry users who want real-time access to data or services from different sources so that they, in turn, can share it in collaboration with their peer group.
One great example of Web 2.0 in action is the Website www.ChicagoCrime.org. This Website incorporates the Google Map API and mixes it up with publicly available crime data from the City of Chicago Police Department's reported crime Website (http://220.127.116.11/). The idea is to provide users information regarding crimes by type, street, data, ZIP code, etc. from the Chicago map. Visitors can switch between viewing regular map, satellite map, and hybrid to information regarding crimes in a particular zone.
What I described above about ChicagoCrime.org is an example of mashup -- a very powerful Web 2.0-enabled Webpage. Mashups are Webpages that take content from many sources including other Webpages to provide the desired content.
Figure 1: A Web 2.0 mashup in practice -- www.chicagoCrime.org
Most Web 2.0 applications have some commonalities, such as:
- Web-based architectural model. Web 2.0 is based on Web architecture that can be classified as a subset of service-oriented architecture. This architecture provides the highly decentralized, globally linked model extensible to the outside world. This model is characterized by technologies like Asynchronous Java and XML (Ajax) and composite applications delivered as mashups. The focus of Web 2.0 towards an open, flexible, participatory model promotes third-party and user collaboration, and intelligence to deliver content. The decentralized content delivery model is designed to reduce application development costs and increase adaptability as well as foster creation of innovative services. Web 2.0 is about flexibility and interoperability. Most Web 2.0 apps are focused more towards consumers using standards like REST (Representational State Transfer), which is simpler than traditional composite application-building protocols like SOAP (Simple Object Access Protocol)
- Cumulative user influence -- The effect of user collaboration and network is critical for Web 2.0. The data acquired from collective and collaborative sources enhances content. For example, Wikipedia entries are created by different people who are knowledgeable about a topic. However, data on a topic acquired from many contributors are compiled neatly using a collaborative approach so that the data does not look haphazard. As services, consumers, and content sources, Web 2.0 applications leverage the Web community as content and application creators providing cumulative knowledge capital.
- Freedom. Last but not the least, Web 2.0 is all about freedom. The quick application development process, ability to share artifacts across geographic borders with strangers through open APIs, developing assets without being confined by a workflow, writing blogs to communicate, using wikis to manage requirements, or the propensity to leverage the Web community to test production-ready code -- all these features of the Web 2.0 environment provide application developers the freedom to be creative without the pressures of an imposed structure.
In some ways, Web 2.0 is Part 2 of the dot-com era. It can also be viewed as a happy-go-lucky version of a global SOA, which focuses on more sharing of services and user collaboration, but maybe less on security and overall management.
A little matchmaking: The case for an SOA-Web 2.0 union
In the last few paragraphs I have explored SOA and Web 2.0 individually. It may seem like SOA is the services solution for big enterprises, whereas Web 2.0 is the way to go for pure Web-based businesses, the carefree kind. However, both of these models can help one another become the services model of the future, because they complement each other in many ways. What is really interesting is the prospect of a SOA evolution, in which SOA incorporates some Web 2.0 best practices. Below are some of the reasons why a combination of SOA and Web 2.0 could help businesses.
- Ability to leverage remote, disparate external services: SOA is primarily focused on optimizing services within the organization, whereas Web 2.0 is a global network of shared services that includes services from inside and outside a company. When internal services of SOA can leverage external services, there will be an opportunity to truly share, interoperate, and build collective assets harvested from a broader intellectual capital base. Today, few serious IT organizations are at a point where they know how to connect to remote services outside their company firewall while guaranteeing data security and integrity.
- Inclusion of social elements in building SOA applications. SOA is robust in configuration control and workflow management; by contrast, today's Web 2.0 apps pride themselves on the lack of control structure. It is common for developers in a Web 2.0 environment to get help for production-ready code from a global resource of developers who contribute through community blogs, wikis, podcasts, etc. to foster internal creative growth among practitioners. For an example, the Rational SOA team actively uses wikis to brainstorm ideas, create draft collateral, etc. for its SOA roadmap.
- Creating best-of-breed apps: I recently attended an innovation forum where one participant from India mentioned how much fun an online auction or sales portal would be if they can added a social element. His definition of social element was to add Webcams and microphones in spaces where sellers and buyers are actually talking to each other, haggling over prices. The underlying auction system could be built using transaction-secure SOA, whereas the human interaction through Webcams would come through Web 2.0 contributions. The point is, adding social elements leveraging Web 2.0 assets and combining that with SOA-centric infrastructure can open up new opportunities for businesses that combine the best breeds of social interaction and composite application. Another example: Shipping companies using a service-based composite application can overlay real-time shipping information over Google maps to show customers the exact location of a shipped product at any given time.
It is not a completely outlandish notion that the SOA and Web 2.0 concepts may merge in the future. Dion Hinchcliffe, editor in chief of AJAX Magazine and a leader in the SOA-Web 2.0 space thinks that the time for that merger is not very far away. Many entries in his extremely popular Web 2.0 blog at SYS-con.com (the blog had 1 million unique reads in the first six months) is dedicated to the topic of a Web 2.0 and SOA union. Analysts across the IT space have voiced their opinions on this topic. John deVadoss, director, architecture strategy at Microsoft, has conveyed in a recent interview with eWeek (http://www.eweek.com/article2/0,1895,1918120,00.asp) his organization's interest in bridging the gap between SOA and Web 2.0.
In an interview with Computerworld New Zealand, IBM's emerging Internet technology group CTO David Boloker thinks that the next generation of Websites will be like gaming applications that will offer high-quality graphics, underlying subsystems, and mashup programs that will serve as the face of situational applications. He believes that Web 2.0 and SOA will be both essential in delivering those commercial Web applications of the future. IBM is also leading "Open Ajax," an initiative that brings open-source Web 2.0 application development to the forefront of the IT industry. This concept is backed by BEA, Borland, the Dojo Foundation, the Eclipse Foundation, Laszlo Systems, Mozilla Corporation, Novell, Openwave Systems, Oracle, Red Hat, Yahoo, Zend, and Zimbra. All this interest suggest that the Web services development space is up for some interesting change, with big application development backers keeping a keen eye on it.
A well-governed service-based delivery model of the future
If the corporate scandals at Enron, WorldCom, and elsewhere over the last few years have taught us anything, it is that corporations need to follow a simple rule: "Do what you say you will do, then show us how and what you did." The accountability and paper trail needed to fulfill this commitment is a big part of the governance story. Since these corporate scandals, governments have made compliance requirements stricter and are forcing many organizations to follow them. Most businesses are working hard to satisfy requirements for SOX, BASEL II, etc. But compliance is only one part of the story. Businesses are also keen on optimizing their operations to survive in today's cutthroat environment.
In short, businesses are not going to cope today unless their applications are deployed under a governance strategy within the sort of adaptable framework that SOA provides. The capability of an SOA to create, manage, and orchestrate services within a governed framework is what makes SOA so desirable. At the same time, I believe that organizations with a workable governance strategy should redefine their approach to incorporate the spontaneous nature of Web 2.0 into tomorrow's SOA environments. It's not such a stretch to imagine these SOA/Web 2.0 governance frameworks as the foundation for services delivery in the future.
See Dion Hinchcliffe, "i-Technology Viewpoint: Is Web 2.0 the Global SOA?" in SOAWebServices Journal, at http://Webservices.sys-con.com/read/164532.htm
See Darryl Taft, "Microsoft Seeks to Bridge Web 2.0 and SOA" in eWeek at http://www.eweek.com/article2/0,1895,1918120,00.asp
See Paul Krill, "Make Way for SOA 2.0" in InfoWorld at http://www.infoworld.com/article/06/05/17/78420_HNsoa20_1.html
See Anirban Dutta, "How Indian Companies Can Benefit from Web 2.0" in NASSCOM, December 2005, at http://www.nasscom.in/upload/48700/How%20Indian%20Tech%20Companies%20can%20benefit%20from%20Web%202.0.pdf
See Ulrika Hedquist, "Web 2.0 is hot -- both today and tomorrow" in ComputerWorld at http://www.computerworld.co.nz/news.nsf/0/CD1AE2E4ACABCFEBCC25717300101C90?OpenDocument
1 The mania for this movie began with an online journal entry by screenwriter Josh Friedman, who documented his work for the movie in his blog. Although Friedman's blog entry may have been part of the movie's publicity, the production unit had no idea how organic this marketing drive would turn out to be. Many other blogs about this movie began popping up. One, SnakesonaBlog.com, which was started by Georgetown University law student Brien Finklestein, morphed into "snakes central" filled with news, rumors, trivia, and much more. Newline Cinema, producers of the movie, actually limited corporate involvement to preserve the organic and spontaneous feeling of the online exchanges.