You have probably heard the term Web services mentioned in the context of technical articles, software product descriptions, or maybe even in conversations with co-workers. Web services seem to be important somehow, but with definitions that can include sentences such as, "XML grammar for defining collections of communication endpoints capable of exchanging messages," you may think that the whole concept is extremely complex and unapproachable.
Fortunately, Web services can be explained in such a way that everyone can understand them as long as we don't dive too deeply into the details of exactly how everything works under the covers. You should make an effort to understand Web services because they (and their related catch phrase Service-Oriented Architecture or SOA) are becoming fairly pervasive concepts across the IT world.
Think of Web services like a car: You don't need to understand exactly how all the pistons and camshafts and fuel injectors work on a deeply technical level to buy a car, to drive a car, or to talk to your friends about cars (unless they are hardcore mechanics, in which case, you do). With Web services, you can understand enough about what they are and how they work to know why they are important to you and your life as an IT professional.
In fact, it is much easier now to work with Web services without manipulating a lot of the underlying technology because software vendors and the open source community have been working hard over the past several years to abstract the nuts and bolts of Web services away from the low-level tasks. This allows you to spend more time plugging components together instead of reading long and detailed specification documents and figuring out how to format an XML message correctly.
This series of articles is geared towards helping Domino developers understand and utilize Web services in IBM Lotus Domino V7.0. This introductory article is broad enough to be useful to anyone with a desire to learn what Web services are. Lotus Domino V7.0 incorporates technologies that make it very easy and convenient for developers to create and to expose Web services, and we will discuss that in much greater detail in the later installments.
For now though, let's talk about what a Web service actually is.
What is a Web service?
Simply put, Web services allow computer applications to communicate with one another in a standard way.
A Web service is an abstract concept -- much like the concept of a conversation between people is a little abstract. A conversation involves two or more people talking to each other, using some kind of language that they both comprehend. The language they use defines the words they use, and how those words are put together into sentences. Conversations usually involve some sort of reply and response, where one person makes a statement or asks a question and the other responds based on what the first person said. The people may be sitting face-to-face, talking on a telephone, or in this day and age, may be sending emails back and forth or using an online chat service.
In any case, the conversation itself has many components, and it may occur in slightly different ways depending upon how many people are involved, which language is being spoken, and which technologies (if any) are used by the people who are talking.
While Web services allow applications to communicate, there are a number of pieces involved, which we touch on throughout this article. But the concept is still similar to that of a conversation with the applications communicating, using a common language and often across a network of some sort. The applications can just as easily be on the same computer as they can be on different computers that are across the world from each other, connected only by an Internet connection and a number of routers and servers in between. The nice part is that the applications and computers don't have to be similar at all. You may have two Microsoft .NET programs talking to each other on a single Windows laptop, or you may have a Java program on an iSeries server in Canada talking to a C++ program on a Linux desktop in China, all using Web services.
The following are the standard technologies typically used in basic Web service communications:
- XML. The language (data format) used by the Web service components
- Simple Object Access Protocol (SOAP). The XML messages that are sent between applications
- Web Services Description Library (WSDL). An XML file that defines how the SOAP messages need to be structured and sent
There is also another standard technology known as Universal Description, Discovery, and Integration (UDDI) that can be used in basic Web service communications. We discuss this towards the end of the article; however, the use of UDDI is optional, and in many Web service implementations, it is not used.
Some terminology: exposing and consuming Web services
Before we go too far down the road of explaining our terms, let's review some Web service terminology.
When we talk about exposing a Web service, we refer to an application that publishes a WSDL file and allows other applications to use a Web service that it provides. Applications that expose Web services are also referred to as providers.
When we talk about consuming a Web service, we mean that an application has called a Web service that is on another machine. Web service consumers are also referred to as clients.
XML: the common language
XML is the language that is used for communication between Web service components. The messages that are traded between applications are formatted as XML and so are the files that define the Web services for consumption. Figure 1 describes the structure of a simple XML file.
Figure 1. Basic XML structure
As you can see, the individual pieces of information in the file (the first name, the last name, and so on) are surrounded by tags enclosed by angle brackets. The first name of John
shows up as
<firstname>John</firstname>. You can also have elements that are contained within other elements, such as the
<person> element, which contains the
There are huge advantages to writing Web services in XML, including:
- XML parsers are very commonplace in all modern programming languages, so applications that are written to interact with Web services do not need to parse the raw XML files directly.
- XML files are text files that are human-readable (in other words, as long as you understand the XML language, you can open an XML file in any text editor and understand the contents of the file). This can be a great help in the debugging process.
- XML provides a way to use any standard character set within a message, so you can write a message in English just as easily as you can write it in German or Japanese.
- XML allows you to use something called namespaces and schemas, which let you predefine the expected structure of a particular named element in the file. For example, you could define a
Price element that is always a floating-point number or a PersonName element that has FirstName and LastName sub-elements that are both strings.
Namespaces also allow different elements with the same name to have different definitions, if necessary. For example, a StockPrice element in one namespace may have a ticker symbol and a price, while a StockPrice element in another namespace may have ticker symbol, price, daily high and low, and 12-month high.
The only real disadvantages to using XML, if you can call them disadvantages, are:
- XML is a strict language, so any problems with formatting within the XML message causes the parsing of the entire message to fail (even if the problems can be easily interpreted or overlooked). However, as long as you use a standard XML library in your programs to generate the XML files (which should be the case with Web service implementations), the library ensures that the message is properly formatted.
- The XML representation of a message is a plain text file, so it is generally much larger than an equivalent representation in another format (such as delimited, binary, or custom formats).
These are minor issues though, and the advantages of XML far outweigh the perceived inconvenience.
SOAP: the messages that are sent
You know that communications between Web service applications are written in XML format, but that only solves half the problem. Applications can parse the messages, but how do they know how to interpret the parsed results?
The specification for how the Web service XML messages are formatted is known as SOAP. It defines the structure of the messages so the applications know how to send and interpret the data. The basic structure of a SOAP message is shown in figure 2.
Figure 2. Basic SOAP message structure
As XML, this can translate into something like the following:
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"> <SOAP-ENV:Body> <ns1:GetStockInfo xmlns:ns1="urn:thisNamespace"> <symbol>FOO</symbol> </ns1:GetStockInfo> </SOAP-ENV:Body> </SOAP-ENV:Envelope>
In its most basic form, you have a SOAP Envelope that contains a SOAP Body, and the Body contains the actual data that is being transmitted. Sometimes there is a SOAP Header (within the Envelope and before the Body) that contains extra information, but the header is optional.
Also, in the case of a problem, the SOAP Body contains error information in the form of a SOAP Fault. The Fault is another XML structure that contains a description of an error, for example:
<SOAP-ENV:Fault> <faultcode>SOAP-ENV:Server</faultcode> <faultstring>Server Error</faultstring> <detail>Database not available</detail> </SOAP-ENV:Fault>
Although a SOAP message is a text file in standard XML format, it is often very difficult to create or interpret a SOAP message manually because there are very specific ways that all namespaces and element attributes need to be defined. Luckily, there are a large number of tools and programming libraries available that can create and interpret SOAP messages for you, so a lot of the complexity is abstracted away by the technology you use. Tools can perform the complex work for us.
WSDL: defining the Web services
Every application that provides a Web service uses a WSDL file to describe which services are available and how they should be called. WSDL is another XML-based format that has a defined structure to allow it to be parsed and understood by applications. The WSDL acronym is often pronounced as "Whiz-dull."
At the heart of a WSDL file, the Web service provider defines the methods that are available for consumption by other applications. For example, a Web service may have a method called GetTemperature that returns the temperature in a given city. The WSDL file lists this method along with information such as:
- The specific name used to make a request to the method (in this case, GetTemperature)
- The parameters that must be used in the request, if any (both the specific names of the parameters and the specified data types)
- The format of the value or values that are returned if the request is processed successfully (again, both the specific value names and data types)
- The URL and protocol that should be used to call the method
There is also information within the file that indicates how the SOAP messages should be formatted, such as the namespaces to use, the order and structure of the parameters, and even what extra information may be required in the SOAP Header element or the HTTP header.
If you have never seen a WSDL file before and you open one to read it, you will probably have a difficult time extracting all the information because the actual structure of the file can become quite complicated. All the information about a method (name, parameters, protocol, and so on) is spread throughout various sections of the file and must be pieced together by a client application before a SOAP message can be constructed. An explanation of the parts of a WSDL file and how they fit together is outside the scope of this article.
However, technology once again comes to the rescue. For the most part, you as a developer do not have to read, parse, and understand the internals of a WSDL file. You can use software tools to obtain this information for you, so all you have to worry about is supplying data to a Web service and using the data that comes back as a response. Not only can you use tools and libraries to do this, but you almost always should. There are a number of exceptions, variations, and complexities to all components that are involved in Web services, and you need to be more concerned with using Web services rather than disassembling and comprehending all the components piece by piece.
Protocols: how messages are sent
One little bit of trivia that we haven't touched on so far is how are these SOAP messages transmitted back and forth?
The answer is that they usually travel across the network (and/or Internet) via HTTP in much the same way that Web pages travel from Web servers to the browser on your workstation. The messages do not have to use HTTP (SMTP is the other major contender in this race, although it's probably a distant second). The protocol that is used by a Web service is explicitly defined in the WSDL file.
The SOAP message protocol specified in the WSDL file is usually HTTP. The SOAP client sends the messages based on the specifics of the protocol to be used.
Other Web service terminology you may hear
We've covered the most common terminology so far, but there are other terms that you may hear mentioned when you talk about Web services.
Applications that make use of Web services are generally considered to be loosely coupled, meaning that the services that allow the program to work are not tied directly to the application nor is the application tied directly to the services. The program could easily make use of a different set of services behind the scenes to provide functionality, and the services themselves wait for an application -- any application -- to call them and await a response.
In the real world, an activity such as going to dinner with your friends could be considered a loosely coupled activity. Several friends talk to each other somehow (face-to-face, phone, email, and so on). They all arrive at a restaurant by whichever means is convenient to them, and after dinner, they each pay for their meal in one of a variety of ways. Details don't really matter, and there are several ways that the activities around the dinner can happen, but in the end, everything comes together to end up at a desired result -- dinner with friends.
On the other hand, an activity like driving your car is more tightly coupled. You have a fixed set of tools that need to be used in a specific way to perform predefined tasks. If you want to back out of your driveway, you have to put the car into reverse and press the accelerator. If you want to make a left turn, you have to turn the steering wheel to the left. You do not have many choices or options about how to make things work because the system as a whole is very precise and the components rely strictly on each other.
UDDI is a standard for providing a catalog of Web services that have been made available by any number of different applications. It is like a phone book of Web service providers. Clients can look up service information in the UDDI registry, and the registry returns the necessary details for connecting to that service.
While UDDI was an important standard in the original Web services definitions, its importance has diminished somewhat because it was an optional component of Web services and, as is often the case with options, a lot of people simply opted not to use it.
Many of the more organized corporate environments that have a lot of internal Web service providers have UDDI registries. In fact, it's a great idea to set up a corporate UDDI site as you publish your own Web services for availability within your company. Besides being an aggregator of available Web services, UDDI makes it very easy to change the provider of a Web service behind the scenes. As long as all clients use UDDI to locate the Web service instead of going directly to the providing server, the SOAP calls are automatically directed to the new provider.
However, it is an optional architectural component and is not strictly required for a Web service implementation.
Web service security
Something you may be wondering about as you read through the descriptions of SOAP and WSDL is security. How do you implement authentication when these services are called, if the provider works with non-public information? Certainly, not all Web services are wide open to the public, are they?
That's a very good question, and unfortunately, there is not a single answer. There are a number of different scenarios that you may have to deal with depending upon your situation, for example:
- Can the SOAP messages be passed as clear text, or do they require encryption?
- Do you need simple name/password authentication or does it need to be persistent and token-based?
- If there are tokens, do they need to be signed by a signing authority for validation, and what's the proper way to include them in a SOAP message?
- What if the client isn't sending SOAP messages directly to the provider, but rather is sending them through an intermediary, such as a message queue or another Web service?
In addition, you can't guarantee that HTTP will always be the protocol used for message exchange, so you can't simply use Web service security on top of existing HTTP security methods and be done with it.
There are several specifications that are meant to address these and other Web service security concerns: WS-Security, WS-Policy, WS-Trust, and WS-Privacy. A number of software vendors and committees have been hashing them out for several years. While not all Web service implementations incorporate all the security specifications, the emerging security standards provide at least a few generally accepted ways of addressing a lot of security concerns.
Middleware and Enterprise Service Bus
There is a larger set of Web services standards that goes along with the security standards that are often lumped together (in quite a large lump) and referred to as the WS-* specifications. Together they address many of the important architectural issues that arise when you start putting together a large number of Web services in an enterprise environment. The standards in WS-* deal with such topics as:
- Quality of service
The reason that you need so many standards is because messaging requirements between the Web service client and server interaction in an enterprise application can be more complex than a simple request/response. For example, how do you ensure that a message actually gets to the provider and back to the client? What if the SOAP request is actually a multi-part transactional request? How do you manage processes that involve Web services that call other Web services? What if an application sends a series of requests that have certain timing requirements?
For the larger software vendors, answering these questions and working with these specifications offer both a challenge and an opportunity. Several vendors have come to market with bundles of Web services middleware products -- often referred to as an Enterprise Service Bus, or ESB -- that provide some or all of the management capabilities to address these sorts of issues. These ESB products generally have additional value as they can tie together a number of Web services from across an organization and provide functionality such as logging and message queuing.
And then there's Service-Oriented Architecture. In a lot of ways, an SOA is simply a combination of everything we've already described: loosely coupled Web services from multiple providers, interoperating under a common set of standards (and perhaps an ESB), and brought together by multiple applications that pull data from the services and present it in a variety of ways.
Because it truly is a software architecture, a tremendous amount of planning and coordination is involved with building an SOA. It's not just a bunch of Web services tossed together and mixed around; it's a real organization of how the services are put together and exposed, what kind of management tools and middleware are utilized, and how the services and the system as a whole are monitored and maintained.
On a higher level, SOA is also a mindset. It requires you to think not of large individual applications that operate independently, but rather of everything as a candidate for using or providing a modular set of functionality that can be exposed and used across the enterprise. Instead of thinking in terms of broad applications, you think in terms of specific, well-defined services, that is Web services.
Why is all this important?
Now you know a little about how Web services work -- a client reads a provider's WSDL file, formats and sends a SOAP message based on the definitions in that file, and receives a SOAP message in response. So why is this important? What's the big deal?
Part of the importance is that there is now a standard way that applications can communicate, regardless of which language they are written in or which platform they run on. In the past, we had either custom data formats that were required by applications (which may or may not be well documented), or we had to deal with API-level functions that may or may not be accessible by other programs written in other languages. The use of XML by all Web service standards means that all services are both accessible and well defined.
In fact, it also frees unrelated applications to easily communicate with each other using a common language. One of the difficulties in working with different technologies from multiple vendors has always been trying to get the programs to talk to each other and to exchange data. Now, as long as all your applications can provide and/or consume a Web service, interoperability between disparate technologies is simplified tremendously.
Another advantage is that Web service clients and providers can easily be located on completely different machines, possibly running on totally different hardware and operating systems, and the communication still works. Applications used to be accessible only by other applications on the same machine, or possibly from other machines using a custom data transmission format. Web service applications need only a network connection and an XML parser.
The really big deal comes from the consequence of all of these factors put together. Because we now have a standard way for different applications to communicate across networks, we can design our own applications differently. Instead of writing monolithic applications that provide all their own functionality to reinvent the wheel, we can now write programs that are much more modular in scope.
For example, instead of writing a single large reporting application that gathers data about several different processes, turns them all into charts, and displays them to a user, we can now perhaps have a dashboard that is simply a frontend to several Web services that provide the functionality for us. The compiled data could be retrieved through calls to one or more Web services, and perhaps even the graphs themselves would result from sending this data to a graphing Web service that accepts the data and returns an image of a pie chart or something similar.
The dashboard goes from being a large standalone program to a simple interface. If we need to add new components, we just call additional services. If we want other chart types, we can call a different charting service. If the dashboard needs to be more interactive with drill-down capabilities or custom sorting, the dashboard can pass the requests back and forth between the user and the appropriate Web services. We can even completely change the services that are called in the background without affecting the users and (as long as the WSDL file doesn't change) without making changes to the dashboard application.
As an IT professional, you may be charged with planning or developing these types of interfaces, or these types of services, or both. An understanding of how the pieces fit together (and indeed, just knowing that there are pieces like this) is an important basis for moving forward with such a project.
It is also fortunate that there are so many tools available to help you provide and consume Web services, so whatever technology you use can do a lot of the heavy lifting for you. In the subsequent parts of this article series, we will discuss how you can use IBM Lotus Domino V7.0 to easily provide Web services to clients or systems.
- developerWorks Lotus article, "Practical Web services in IBM Lotus Domino 7: Writing and testing simple Web services"
- developerWorks Lotus article, "Practical Web services in IBM Lotus Domino 7: Writing complex Web services"
- developerWorks Lotus article, "Lotus Notes/Domino 7 Web Services"
- developerWorks Lotus article, "Consuming Web services from a Lotus Domino Java agent"
- developerWorks tutorial, " Quickly create Domino Web services: New Web services function in Domino 7 speeds development"
- developerWorks SOA and Web services zone Standards and Web services page
- developerWorks article, "Which style of WSDL should I use?"
- developerWorks SOA and Web Services Zone
- developerWorks article, "Insight and outlook, Part 1: Why and when should you choose SOA?"
- developerWorks article, "Service Oriented Architecture (SOA): an interview with IBM Workplace and Lotus developers"
Get products and technologies
- Download a trial version of Lotus Domino from developerWorks.
- Download a trial version of Lotus Notes from developerWorks.
- Participate in the discussion forum.
- Visit the developerWorks blogs and join the developerWorks community.