When is an airport ever really finished?
Building Internet and intranet applications is much like building an airport. The expansions and changes needed to keep up with passengers, airlines, and facilities prove that perpetual motion may indeed exist. Web sites normally expand in the same way; the basic Web system built last year now needs a new set of functions. Integrating new functions and new server software is facilitated by SOAP, a new open-standard enabling a previously unrealized level of integration that used to require an expensive, unmaintainable engineering effort.
This case study shows how Web services were used to provide a seamless and easy single-sign-on experience for users. You will see how a development team used Web services to seamlessly integrate an eCRM application into an existing e-Commerce marketplace application. I show detailed examples from the experience Switchouse had integrating Inclusion collaborative messaging software into its Web site. The article also presents several problems encountered during the integration project that I hope you will avoid.
Reducing expenses for an online consumer marketplace
Switchouse is one of the leading consumer marketplaces on the Web-focused on bringing together millions of people to help them find new and better ways to get things they want. Unlike eBay, the popular auction site, Switchouse enables users to buy and sell second-hand entertainment items such as music CDs, movies, video games and books at fixed prices. Switchouse earns a fee for each transaction. The typical purchase decision is in the $10 - $15 range. Switchouse needed an inexpensive way to support their users.
Shortly after launching the e-Commerce Web site, Switchouse management decided to add on-line customer relationship management (eCRM) services to reduce the cost of supporting users. Switchouse uses the eCommunity portion of its Web site to provide a member-to-member support system.
Switchouse chose to outsource their eCRM software needs to Inclusion for its public and private discussion groups, e-mail announcements, and notifications to systems integrator, Inclusion, Inc. of Los Altos, CA. Inclusion software features a tag-based scripting language to match the look and feel of their script-generated pages to that of Switchouse's e-Commerce site pages.
Figure 1. The Switchouse.com home page offers sign-in forms.
Figure 1b. Inclusion-generated eCRM pages offer sign-in forms.
Inclusion software provides a single-sign-in Web service called GateKeeper. Users sign-in from either the Switchouse e-Commerce pages or the Inclusion-generated pages. Switchouse plans to add additional features and use the same single-sign-in Web service to integrate these too.
The primary concern over integrating the systems would be to give the user a good experience. Experience showed that Switchouse users have little tolerance for slow response times, so the sign-in system needs to work in real-time. Additionally Switchouse had these requirements:
- The user is only challenged once during a session for an id and password.
- Users may sign-in from any page.
- Sign-in service operates in real-time. Once a user clicks the sign-in button the response may take no longer than 5 seconds.
- Inter-server communication is secure and authenticated.
- System interoperates bidirectionally.
- System is extensible to provide a generic way to share data.
Two things concerned the design team. How will the system immediately respond to the sign-in form on the Inclusion generated pages? Secondly, will the system scale when a larger volume of data needs to be transferred sometime in the future.
The solution was implemented in two parts. A synchronous protocol uses HTTP POST and redirect commands to make the sign-in appear to take place in real time. A second, asynchronous protocol handles transferring additional data. I'll explain the synchronous protocol first.
Figure 2. The synchronous portion of the GateKeeper system.
The script-generated pages include an HTML form to sign-in. The form posts to the KeyMaster servlet on the e-Commerce application server. The return parameter tells the KeyMaster where to redirect the user once the sign-in finishes successfully.
Listing 1: HTML sign-on form
<FORM action="http://www.switchouse.com/keymaster.html" METHOD="POST"> <INPUT type="text" size="10" maxlength="32" name="login_username"> <INPUT type="password" size="10" maxlength="32" name="login_password" <input type="hidden" name="return" value="/switchouse/summary.html"> <input type="hidden" name="submitKey" value="Log In"> </FORM> |
The KeyMaster servlet on the e-Commerce application server validates the user account and responds to the form request with a redirect back to the GateKeeper servlet on the eCRM server. The redirect looks like this:
http://eCRMsite.inclusion.net/sq=%0A%12%38%31...%8B%9C&return=/switchouse/summary.html |
The sq parameter contains the user data. In a SOAP-based Web service environment the decrypted user data looks like this:
<gatekeeper> <name>Frank</name> <password>83jdiej</password> <action>sign-in</action> </gatekeeper> |
The GateKeeper servlet decrypts the user data, signs-in the user to the eCRM application, and redirects the user's browser to the return page. Though there are redirected pages in this transaction, the user only sees the original page with the sign-in form, and the return page with the user signed-in. As a result the user is happy.
To meet the goal of a bidirectional interchange between the e-Commerce and eCRM servers, the GateKeeper implements an asynchronous protocol. Switchouse uses the asynchronous Web service when new users register with the e-Commerce site.
New user registrations typically include name, address, phone number, group affiliation, and permissions as a new member. The synchronous method is not guaranteed to work if the URL-encoded user data grows to be longer than 2000 bytes -- many browsers clip very long URLs. The GateKeeper uses the synchronous Web service to receive the most basic user information and the asynchronous Web service to receiving the additional user information.
New users enter registration information -- name, address, and more -- into an HTML form. The form posts to the e-Commerce server causing the new user data to be inserted into the e-Commerce system database. The post returns a redirect command, which includes a GateKeeper XML document.
<gatekeeper> <name>Frank</name> <password>83jdiej</password> <action>new-user</action> </gatekeeper> |
The GateKeeper servlet decrypts the XML document, adds the new user to the eCRM server database, signs-in the user, and redirects the user to the return page. All of this appears in real-time to the user.
The GateKeeper also adds an entry into a queue of asynchronous requests. The entry instructs the GateKeeper to contact the e-Commerce server KeyMaster servlet to retrieve the user's extended information, including company, title, phone number, department name, managers name, and e-mail address.
<keymaster> <name>Frank</name> <password>83jdiej</password> <action>send-extended-info</action> </keymaster> |
The KeyMaster makes a Web service request to the GateKeeper with the extended user information.
<gatekeeper> <name>Frank</name> <password>83jdiej</password> <action>update-user-info</action> <company>PushToTest.com</company> <title>Developer</title> <phone>408 374 7426</phone> <department>7</department> <manager>Fred Gibbons</manager> <email>fcohen@pushtotest.com</email> </gatekeeper> |
The same system works for exchanging data for new users, updates to existing users, deleting users, and reporting errors. For example, the response XML document may be an error message.
<gatekeeper>
<action>error</action>
<error>400></error>
<error_description>KeyMaster received an XML document that
will not decode</error_description>
<timestamp>38274883</timestamp>
</gatekeeper>
|
With the synchronous and asynchronous Web services, the system achieves the single-sign-in goals. The new user has immediate access to the e-Commerce site. The measured latency for the asynchronous system is usually less than five seconds. The user is happy.
Lessons only developers should have to learn
I would have enjoyed designing the single-sign-in system, implementing the Web services and doing the occasional maintenance every once in a while. Back in the real world I encountered several problems which needed to be solved.
With an imminent launch of a new product line, a manufacturer of electronic equipment choose the Inclusion software to build a secure extranet to distribute product documents and receive feedback from their field sales managers. The manufacturer wrote the KeyMaster servlet to authenticate users against an internal LDAP directory of employees.
To meet the launch schedule, the manufacturer chose Inclusion's hosted Application Service Provider (ASP) service to get started and at a later date to roll the Inclusion software into their datacenter. The KeyMaster and LDAP directory reside inside the manufacturers network and the Inclusion software resides on an external network. Unfortunately, while the manufacturer's network allows outbound HTTP requests the network blocks inbound HTTP requests.
Figure 3. The unidirectional firewall.
When a new user signs-in, the KeyMaster sends the URL encoded request to the GateKeeper. The GateKeeper then tries to make a Web service request to the KeyMaster to request the additional user information. The GateKeeper request receives a redirect command to a SecureID-based authentication system the manufacturer uses to protect its network.
The bi-directional nature of the single-sign-in system failed to cross the firewall. At first I considered asking the manufacturer's IT group to open a special unprotected connection to the internal KeyMaster server. I rejected this idea when I put myself in their shoes and considered if I would want your own network opened up for special cases.
The solution is to change the call-flow to enable only the KeyMaster to initiate transactions. The KeyMaster running on the manufacturer's network would contact the GateKeeper when a new user registers and when a user signs-in.
Six companies have now used the GateKeeper single-sign-in system. Inclusion found a trend emerging early on: The single-sign-in project requires the highest resourcing and planning to be implemented on time. Developing a Web service requires knowledge in basic coding, XML parsing, and system integration with directory services, including LDAP services. Few engineers have all of those skills. From my experience, a mid-level engineer comes up to speed on the single-sign-in Web service in three-five weeks. Code delivery usually comes in four weeks with two additional week of testing required.
The single-sign-in Web services described in this article also plugs into HTTPS secure protocols. The GateKeeper and KeyMaster protocols operate over secure SSL connections. The browser and Web server software normally take care of providing SSL encryption and authentication.
The current SOAP 2.2 specification does not define security and authentication protocols. However, many engineers expect the eventual SOAP specification to define SSL as a means to securely make Web service requests. OpenSSL.org is a good source of SSL technology, and it's free.
DTD today, XML schema tomorrow
The single-sign-in system presented here exchanges XML documents during Web services calls. The system was designed in 1998, long before XML libraries (SAX and DOM parsers) were widely available. Initial implementations were in Java and used SAX libraries from IBM alphaWorks. While powerful, SAX is also difficult for mid-level software developers to use. Many callback methods must be implemented which require a thorough understanding of XML and Document Type Definition (DTD) syntax.
The next generation single-sign-in system will use SOAP and XML Schema to define the parameters of the request and response calls. The Java implementations will JDOM for creating and manipulating the XML documents. JDOM is a simple API that runs over SAX and DOM parsers to provide a more Java-like method of working with XML data.
I am the KeyMaster. No, I am the KeyMaster
In my experience, engineers completing their first server software integration project successfully causes many more integration projects to follow. The world seems a little easier. The GateKeeper/KeyMaster Web services found use in adding promotional tie-ins with credit card companies, integration with incentive programs (like airline mileage points systems) and survey systems.
At first this presented a potential problem: As an e-Commerce company wanting to integrate another company's service into your system, what happens when the other company offers their own server-to-server communication system? Who is the GateKeeper and who is the KeyMaster? Web services solves this dilemma by making all systems both a client and a server. Inbound requests are transported by the same protocols as the outbound responses.
In many modern server systems interoperability starts with access to a common directory to provide users with a secure single sign-in mechanism between systems. Interoperability then spreads to data synchronization, reporting and e-Commerce. By implementing synchronous and asynchronous Web services-based server-to-server communications protocol, users are happy and the resulting software is maintainable.
- Load, a free open-source utility for testing SOAP-based Web services for scalability and performance.
- Web services Definition Language (WSDL) specification.
- SOAP protocol specification.
- A list of publicly available Web services.
- Check out the look and feel of Switchouse.com.
Frank Cohen is a software entrepreneur who has contributed to the worldwide success of personal computers since 1975. He began by writing operating systems for microcomputers, helped to establish video games as an industry, helped establish the Norton Utilities franchise, led Apple's efforts into middleware and Internet technologies, and most recently was principal architect for the Sun Community Server, Inclusion.net, and TuneUp.com. Frank maintains Load and operates PushToTest, a scalability and performance testing solutions company. More information is available at www.PushToTest.com.You can contact Frank at fcohen@pushtotest.com.
Comments (Undergoing maintenance)
Table of contents
Next steps from IBM
Rational Tester for SOA Quality is an automated web services testing and SOA testing tool that supports Web services SOAP, HTTP(S), JMS, WS-Security, UDDI, and RESTful standards.
- Try: The no-charge Rational Service Tester for SOA Quality helps quality assurance professionals validate web service based SOA application..
- Article: See the whats new with Rational Service Tester for SOA Quality, and its Generic Service Client, which provides a single client to interact with any SOA service type.
- Tutorial: Get hands-on experience with this Transport for SOAP feature tutorial using Rational Tester for SOA Quality.
- Buy: Rational Service Tester for SOA Quality
My developerWorks community
Tags
Use the slider bar to see more or fewer tags.
Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere).
My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).
Popular article tags |
My article tagsSkip to tags list
Popular article tags |
My article tags
