IBM Lotus Instant Messaging and Web Conferencing provides enterprise user communities with real-time collaboration features such as presence awareness (the ability to determine whether or not a person or application is online and available), instant messaging/chat, and Web conferencing. Lotus Instant Messaging 6.5.1 (formerly Sametime) is offered as a stand-alone product. It can also be tightly integrated with other IBM technologies, such as Lotus Notes/Domino and IBM Workplace. (Starting with release 6.5.1, Lotus Instant Messaging is shipped on the same release schedule and version as Lotus Notes and Domino with a common set of supported operating systems, languages, and browsers.)
For many users, Lotus Instant Messaging has become one of the most valuable and frequently used tools within their organization -- so indispensable, in fact, that many people want to extend this capability to interact with other Lotus Instant Messaging user communities outside of their companies. If this sounds like an interesting idea to you, read on.
In this article, we explain how we set up connections between separate Lotus Instant Messaging communities. In the first example, we connect two Lotus Instant Messaging communities within IBM. In the second example, we connect an external Lotus Instant Messaging community to IBM's internal Lotus Instant Messaging environment. This external environment belongs to a large IBM customer. We created these connections by using the SIMPLE (SIP for Instant Messaging and Presence Leveraging Extensions) protocol. SIMPLE is an add-on to SIP (Session Initiation Protocol), a standard defined by the Internet Engineering Task Force (IETF). Some industry insiders predict SIMPLE will form the basis for a new Instant Messaging and Presence Protocol (IMPP). SIP was originally developed for voice over IP (VoIP), but has since incorporated support for Web conferencing, live video, and other media. SIMPLE is backed by Microsoft, IBM, Sun, Novell, and other industry leaders.
At the conclusion of this article, we offer a sneak peek at upcoming functionality that will allow you to extend your Lotus Instant Messaging community to include Web users -- no instant messaging clients required! Our goal is to share this knowledge with you to help you connect your own SIP-compliant instant messaging communities with others. This article assumes that you're an experienced system administrator familiar with Lotus Instant Messaging. For more information, see the developerWorks: Lotus article, "SIPing with Sametime."
Connecting two internal IBM communities
IBM's primary Lotus Instant Messaging community consists of over 250,000 clients. These clients connect to a central WebSphere Edge Server and 10 multiplexor (MUX) servers that disperse chat sessions between four Community servers. These four servers comprise IBM's internal messaging.ibm.com community cluster. For more information about IBM's Sametime implementation, see the articles, "The hitchhiker's guide to Sametime deployment at IBM" and "Life in the fast lane: IBM moves to Sametime 3."
Our initial SIP/SIMPLE installation involved connecting IBM's primary Lotus Instant Messaging community to a smaller, separate community of users within IBM. (This smaller community consisted of users developing and supporting the Lotus Software brand.) After we succeeded in connecting these two internal communities, we extended our connection to include Lotus Instant Messaging to a customer site. This external community included approximately 10,000 instant messaging clients. (Plans are to further expand our virtual community to include additional customer sites in the near future. This includes using SIP/SIMPLE to connect an external Community server outside the firewall, allowing external Internet customers to connect to IBM users.)
SIP Gateway and SIP Connector
To connect multiple communities, Lotus Instant Messaging uses the SIP Connector and SIP Gateway. The SIP Gateway (which consists of the file stgateway.exe) connects the Community server to the SIP Connector. In Lotus Instant Messaging 6.5.1, stgateway.exe is natively installed with other files on the Community server. After installation, you must activate the SIP Gateway by setting the Support External Communities setting in the Community Gateway document to True. When you activate the SIP Gateway, Lotus Instant Messaging users can add external users to their contact lists. (If the SIP Gateway is not configured properly, users will not see the option to add external users.)
The SIP Gateway is usually located on the Community server, but in theory can also be located on a separate system, much like a MUX and Connector system. But because the SIP Gateway must communicate directly with the Community server, the out-of-the-box configuration of stgateway.exe running on the Community server configuration is easier to administer in a clustered environment.
Each Lotus Instant Messaging community requires a separate SIP Connector to connect to other Instant Messaging communities. The SIP Connector communicates with the local Community server's SIP Gateway on port 1516 and communicates with the remote SIP Connector on ports 5060 and 5061. The SIP Connector is simple to install and can be found on CD2 of the Lotus Instant Messaging installation media. You must install the SIP Connector on a system that is separate from the Community server (see figure 1):
Figure 1. SIP Connector and SIP Gateway architecture
Both the Lotus and main IBM communities are made up of multiple Community servers that are clustered together via Lotus Instant Messaging. One server on each cluster provides the interface to a SIP Connector, which connects the two communities together. The SIP Connector pulls its configuration information from the Community server by setting the ConnectorName and VPS_HOST parameters in the Sametime.ini file. The ConnectorName value must match the Connector Name value in the Community Connector document in the Community server's stconfig.nsf file. The server that the connector gets its configuration from is defined in the VPS_HOST value.
For more information about installing and configuring the SIP Gateway and SIP Connector, see the product documentation.
The following is an example section of the Sametime.ini file on the SIP Connector. If you enter incorrect values for one of these Sametime.ini parameters, you see errors in the sametime.log file where the connector cannot make a connection to the Community server and continually attempts to start:
To configure SIP/SIMPLE for an existing Lotus Instant Messaging Community server, you must configure several documents in the stconfig.nsf file.
Community Connector document
The Community Connector document specifies the name of the local connector, its IP address, and port. We configured the TLS Encryption option between the communities by setting the connector's IP address and port of the local connector in the TLS IP and TLS Port fields. If the IP field is blank and the port is 0, the SIP Connector will not listen to make unencrypted connections. The Supported Communities field will match this connector with the remote connector that is specified in the Community name field of the External Community document (described later in this article):
[Connector Name: Connector1
TLS IP: x.x.x.xx
TLS Port: 5061
Supported Communities: Customer1
Community Connectivity document
The Community Connectivity document specifies the IP address of the local connector. The address of the local connector must be added to the local Community Trusted IPS field:
[Community Trusted IPS x.x.x.x;
Community Gateway document
In the Community Gateway document, you can enable and disable the SIP Gateway. Setting the Support External Communities field to True enables the SIP Gateway. If set to False, SIP is disabled, and clients cannot see the option to add external users.
External Community document
The External Community document contains the information for the remote community. The Community Name field in this document must match the Supported communities value in the Community Connector document. The Domain fields present the extension name that you want SIP/SIMPLE to route. And the DNS and port identifies the DNS name (or IP address) and its listening port of the remote SIP Connector. We configured the TLS Encryption option by setting the port to 5061 and the Encryption field to Mandatory:
[Community Name: Customer1
Certificate distinguish name:
Connecting external communities
After our internal test was complete and our IBM and Lotus communities were successfully connected, we took on the task of connecting our internal community with an external customer community. To connect this community with IBM's, we placed the SIP Connector outside IBM's corporate firewall. The firewall passed port 1516 from the connector to our Community server. The connector also passed ports 5060 and 5061 through firewalls between the connectors. We installed the connections using the same procedure described in the previous section of this article.
In general, network IP address translation did not cause any issues with SIP/SIMPLE. The IBM Community server-to-connector firewall and the customer's firewall between the connector and the Internet both used network address translation configurations.
SIP/SIMPLE requires that the Lotus Instant Messaging clients be Sametime 3.1/Lotus Instant Messaging 6.5.1 or later. After the SIP configuration is set up properly on the Community server, clients who log into the SIP-enabled Community server can see a new external option to add SIP users to their contact lists. When the external SIP user becomes active, the Lotus Instant Messaging client distinguishes the SIP user by displaying a tiny green square with a globe icon next to the SIP user's name in the contact list. Usually the company name is chosen as a group name to organize external SIP users in the contact list.
In our pilot, users experienced a few functional differences when adding a SIP user to their contact lists compared to adding a local (Lotus Instant Messaging) user. Local users cannot add an invalid user. However, SIP users (similar to users of most other public instant messaging networks) can add an invalid user because Lotus Instant Messaging cannot provide external name lookups when adding a SIP user to the contact list. So if you enter an invalid name (for example, by misspelling), that invalid name appears off-line forever. Because Lotus Instant Messaging does not have access to the external domain's LDAP directory, it cannot provide name lookup functionality.
Testing the connection
After we set up the SIP connection, we performed several tests to verify whether or not it was working correctly. You can use these same tests to determine whether or not your own connection is working.
Client Message notification
You can customize the message that is displayed on the user's chat session when a message cannot be sent by the stgateway process (more on this process later in this article). To do this, edit the VP_IM_FAILED_MESSAGE parameter in the Stgateway.ini file:
VP_IM_FAILED_MESSAGE = This message is sent by the SIP Gateway. The last message you sent was not delivered. The user may be offline or there may be a system problem. Please verify that the user is online before contacting the system administrator.
Transport Layer Security (TLS) is an option that can be configured on each connector to provide end-to-end security by encrypting chat messages. (Lotus Instant Messaging provides this feature out-of-the-box, but you must specifically configure encryption over SIP/SIMPLE.) To assist in troubleshooting our TLS configuration link to our customer, we created a directory called Trace and temporarily added the following lines in the SIP Connector's Sametime.ini file to enable debugging:
MESSAGE_DEBUG = 1
Trace files are created in the Trace directory. These files provide some primitive, but helpful information to show that the connectors are speaking with TLS. Make sure to remove these debug parameters for production.
Another way you can verify whether or not the TLS ports are accessible is to telnet to ports 5060 and 5061 from one connector to another. Once connected, you will see a blank screen. If you do not connect, the ports may not be enabled through the firewall. After TLS is established, you can see port 5061 on the connector when doing a netstat command.
Earlier versions of the stconnector and stgateway files (see the following section) experienced some configuration and performance problems when configuring TLS, but after we installed stconnector.exe 220.127.116.11, stgateway.exe 18.104.22.168, and the latest updated files on the Community server, TLS encryption functioned properly.
To make things easier for configuring TLS, we first set up the SIP connection without TLS. Then after we verified that both communities were working fine, we set up the functional link with TLS. To configure without TLS, open the Community Connector document and remove the TLS IP and TLS Port fields and set the IP field to the connector's IP address and the port to 5060. Also set the port in the External Community document to 5060.
The stgateway.exe gateway process makes the SIP user appear as a local Lotus Instant Messaging Community server's user by providing contact list updates on the SIP user's online activity status. However, if stgateway.exe is overworked, it may not be able to keep up with the large amounts of SIP contact list updates. In this case, the local users may not see immediate online awareness of their SIP contact list users. Earlier versions of stgateway.exe and the SIP Connector had long delays in awareness updates, but these delays have been addressed in the latest versions via hotfix TMAY-66UQKZ. (You can obtain this hotfix from IBM Support.) If you don't have the hotfix installed, your users can work around the SIP awareness update notification problems by logging off and logging on again. This causes stgateway.exe to provide an updated awareness status and to repaint the users' SIP contact list awareness status.
We successfully tested the option to move stgateway.exe off the Community server and to install it on the SIP Connector. But we have found that even though we installed stgateway.exe on the stconnector system, all traffic still must go through the Community server before transferring between the gateway module and the connector. Thus, there is no overall benefit of running stgateway.exe on another system. Unless performance on the Community server is peaking, it is more efficient with network traffic to leave stgateway.exe on the Community server as is the default configuration.
Testing Community server and SIP Connector system performance
After we installed our SIP connection and made sure it was working, we determined what kind of performance our users could expect when communicating with members of the remote community. To do this, we conducted "burst workload" tests by using Rational test scripts to simulate 3,000 users simultaneously changing contact list awareness status or pressing Enter on a chat. (In real-world environments, it's probably unlikely that most communities will have this magnitude of instantaneous activity.)
During these tests, the SIP Connector and Community servers consistently averaged less than 50 percent CPU usage. Memory consumption was also very stable and did not increase on the SIP Connector and Community servers.
CPU utilization did not appear to be a problem on our tests and initial deployments. If CPU utilization is a concern in your environment, you can examine configuration options to minimize Community server CPU. For instance, you can remove the stmux.exe process from the Community server and install it on a separate MUX system. With that change, the entire user session overhead is removed from the Community server.
We tested the option of also running the stgateway process on the same system. You can reduce the CPU overhead caused by stgateway.exe, but you will see an impact of doubling network I/O between stgateway.exe, the Community server, and stconnector.exe. Therefore, this configuration option should be investigated if your Community server's CPU is high, and you have plenty of network I/O capacity.
SIP response time performance
Before adding SIP in our production domain, we ran response time performance tests. We used Rational Test Manager to drive test workload scripts to the Stress Tester toolset. The Stress Tester is an IBM-internal Lotus Instant Messaging testing tool that logs into the Community server and issues command line sequences that can mimic thousands of instant messaging clients changing contact list online awareness status or chatting with other users. The Rational system was used to send these test commands to the Stress Tester and to collect the response time results.
We also measured the Lotus Instant Messaging communities' chat response times before and after adding SIP. In our test environment, our local Community server logged in 9,000 Lotus Instant Messaging users that connected by SIP to a 3,000 user remote community. We then measured chat and online awareness response time for:
- 3,000 local users sending to another 3,000 local users
- 3,000 local users sending across SIP to 3,000 remotely connected SIP users logged into the remote SIP community
Response times were fairly consistent for contact list online awareness and chat instant messaging. As expected, local users' response times were faster than SIP users' response times for both chat and contact list online awareness tests (0.85 second versus 3.25 seconds).
Adding SIP users to our test Community server did not cause any significant response time delay to existing users. The response times for 3,000 local users sending chat messages only changed minimally (from 0.74 second to 0.85 second) when an additional 3,000 local users communicating to 3,000 SIP users were added to the test. These test results indicate that adding SIP users to the Community server does not present a performance concern for existing local Community server users' response times.
Our internal test SIP connection has now been running for several months, and we have not seen any degradation of performance when adding additional SIP users to our Community servers. One final point: When adding new SIP users, be sure to configure the connection to the new SIP community properly. In one case, when incorrectly adding a SIP configuration, we experienced intermittent problems with other SIP links that were on the same Lotus Instant Messaging Community server. Therefore, we recommend that you add new SIP links on your production cluster only during off-hours.
Extending your Lotus Instant Messaging community to Web users
Connecting separate communities together via SIP/SIMPLE significantly broadens the reach of Lotus Instant Messaging. IBM plans to pilot several new features that will extend this capability even further and make it possible for Web browser users to chat with our Lotus Instant Messaging community via HTTP. This will involve installing an external Lotus Instant Messaging cluster outside IBM's firewall that customers can connect to. We will use SIP to connect this external instant messaging community to IBM's community.
Browser clients can chat with IBM employees via a URL link that will be made available through the Collaborate Now Web site. These users will click the URL and be routed to a WebSphere Edge Server load balancer that will evenly distribute their connections to several Lotus Instant Messaging MUX systems. As this pilot becomes operational, we will share our experiences with you as a guide to set up your own Lotus Instant Messaging environments to allow Web browsers to chat with your users.
When users are routed to the MUX, their LTPA tokens will be passed from the Collaborate Now site to the Lotus Instant Messaging Java connect client so that their clients are automatically authenticated with the Community server. The Lotus Instant Messaging Java connect client will automatically be downloaded to the client, and their contact list of IBM employees will appear. (We will use out-of-the-box Lotus Instant Messaging for these browser clients.)
To download the Java applet, users should configure their browser settings to allow popups and the ability to download Java applets to the local machine. To enable the Java clients to connect using HTTP over port 80, a special Lotus Instant Messaging server fix was created and is available from Lotus Support.
IBM will control the number of connections that we allow to log on to our community by adding the parameter vpmx_capacity to the MUX's Sametime.ini file. Clients will receive an "Unable to access -- try again later" message when they reach the maximum number of connections allotted by the MUX. In the sametime.log file, the administrator will see the message "Full - reject new connections."
How to use SIP/SIMPLE to route users with different mail extensions
To make the architecture work, we had to overcome an issue in which SIP requires a common mail extension. SIP routes between communities based on the user's mail extension, for example, IBM.com on one side and Customer1.com on the other. Because people are from different companies, SIP requires a common mail extension to provide routing from one side to another. To accomplish this, IBM created a custom Java class file. This file can be referenced in the LDAP section of the stconfig.nsf file on the Lotus Instant Messaging server. The custom class file converts all user IDs to a common mail extension that SIP can use to route users.
The class file also eliminates wildcard searches that keep the Customer1.com users from adding users without knowing their exact names or from adding the entire company to their contact list. This improves performance and provides a level of security.
Installing the required custom LDAP Java class file
After you develop a custom Java class file, you must:
- Place it in the Lotus Instant Messaging c:\lotus\domino directory.
- Add the ST_JAVA_CLASS_PATH= c:\lotus\domino flag in the [config] section of the Sametime.ini file.
- Modify the following in the LDAPServer document in the stconfig.nsf file:
Search filter for resolving person names: StLdapCustomized.externalSearchFilter()
Attribute of the person entry that defines the personâs email address: StLdapCustomized.externalEmail(mail)
Note: For the custom Java class file to work correctly, you must run with StResolve.dll 22.214.171.124 and StLdap.dll 126.96.36.199 on the Lotus Instant Messaging server.
For more information about customizing LDAP search filters in Lotus Instant Messaging, see the developerWorks: Lotus article, "Customizing LDAP directory searches in Lotus Instant Messaging 6.5.1."
This article has shown how SIP/SIMPLE can be an important tool for connecting Lotus Instant Messaging communities both internally and externally. And it has the further potential for extending your instant messaging community to Web users, giving your community a truly global reach. We hope that by sharing our experiences with you that we can help prepare you to use SIP/SIMPLE to extend your own Lotus Instant Messaging community.
- For more information about IBM Lotus Instant Messaging and Web Conferencing, visit its product page.
- For more information about using SIP with Lotus Instant Messaging (Sametime), see the developerWorks: Lotus article, "SIPing with Sametime."
- See the articles, "The hitchhiker's guide to Sametime deployment at IBM" and "Life in the fast lane: IBM moves to Sametime 3," for more information about IBM's Lotus Instant Messaging (Sametime) implementation.
- Consult the product documentation for details about installing and configuring the SIP Gateway and SIP Connector.
- Get involved in the developerWorks community by participating in developerWorks blogs.
- Purchase Lotus books at discounted prices in the Lotus section of the Developer Bookstore.
Dig deeper into IBM collaboration and social software on developerWorks
Get samples, articles, product docs, and community resources to help build, deploy, and manage your cloud apps.
Experiment with new directions in software development.
Software development in the cloud. Register today to create a project.
Evaluate IBM software and solutions, and transform challenges into opportunities.