What is new in Lotus Enterprise Integrator (LEI) 7
IBM Lotus Enterprise Integrator for Domino (LEI) allows users to access data across multiple platforms. With LEI, users can work with corporate data, irrespective of where the data actually resides. This lets you leverage your existing store of company knowledge, without having to port this information to a common platform or application. LEI consists of two major components, the LEI Server and the LEI Administrator. The LEI Server monitors the LEI Administrator database for LEI activities to execute. The LEI Administrator (a Notes application) lets users create activities and connections.
IBM recently released Lotus Enterprise Integrator 7, the latest version of LEI. This article gives you a short tour of the enhancements made in LEI 7. These focus on the following two areas:
- Reliability and Serviceability (RAS) enhancements. In LEI 7, these include support for Domino cluster failover, Batch and RealTime activities, and NSD changes.
- New features and user interface changes, including Lotus Connector for Oracle improvements.
This article assumes you're familiar with previous versions of LEI. For more information about LEI, see the Lotus Enterprise Integrator product page.
An important RAS change in LEI 7 is support for the Domino cluster. In previous releases, an LEI cluster worked like this: A group of LEI servers could participate in an LEI cluster. You could pick which server in the cluster that an activity would run on by default. If that server went down, another LEI server would pick up that activity and run it. Each server in the LEI cluster used the same LEI Administration database to work with. This last point meant that if the Domino server that hosts the one LEI Administration database went down, then all LEI servers in the cluster went down. In other words, the LEI cluster had a single point of failure.
This is no longer the case. LEI 7 can now be installed on each server in a Domino cluster. Each Domino server will have a cluster replica of both the LEI Administration database and the LEI Log database. So, if any one Domino server goes down, the other LEI servers on the other Domino cluster servers will pick up the work of the inactive server (which they know about in their local replicas).
Setting this up is a breeze. You first find out about the feature in the LEI 7 installer. During the install process, you will see the screen shown in figure 1:
Figure 1. Installing LEI Failover Cluster support screen
The installer will either create the first server of the cluster, or join an existing cluster.
In the LEI Administration database, in the Configuration document, we will see the Domino Clustering field enabled and other cluster-related fields properly configured by the installer (see figure 2):
Figure 2. Domino clustering enabled
As figure 2 shows, below the Domino Clustering field is the Synchronization Delay field. A typical cluster replication is around eight seconds, and then we allow a minute more than that in case system clocks wander a bit. To check the time on a system clock, run the LEI Server Administration\Server Time Check Action from the Actions Menu. You will get a result like that shown in figure 3:
Figure 3. Server Time Check Action result
If times are too far apart among your servers, you need to go to each server and synchronize them. There are many ways to have the systems stay on Atomic time by themselves. We recommend that you find the correct time synch tool for your operating system. It will make life easier.
The final configuration field is the Batch Activity Failover field (see figure 2). It lets LEI fail over these types of activities (everything except RealTime). The following section explains Batch and RealTime activities.
Batch and RealTime activities
LEI 7 offers two types of activities, Batch and RealTime.
Batch (or Data Management) activities are the only type of activities to actually fail over. This requires that each server in the cluster has identical relational database management system (RDBMS) client configurations (or in the case of the Lotus Connector for SAP, that the SAP Connector be installed on each server in the cluster). The activity author can define the order of failover. For example, always run an activity on server HR1, and if that is down, use HR2. This is done in the Activity Execution Options section of the activity documents (see figure 4):
Figure 4. Activity Execution Options screen
After the primary server comes back up, the activity will start running there again.
The design of LEI RealTime activities requires that they be handled differently in the LEI failover cluster. First, let's look at the Virtual Fields activity. In this case, each LEI server in the Domino cluster must have its own Virtual Fields activity, to run locally. They may all share the same RDBMS table. The Virtual Field activity is smart enough to know that if a record is changed on cluster server A, it does not need to be changed on any other server in the cluster (as long as a Virtual Fields activity is running for that database and form on the other servers). The easiest way to set this up is to create the Virtual Fields activity once, and then copy and paste that activity document once for every server in the cluster. Then edit each activity and set the correct designated server field.
Virtual Documents is significantly different from Virtual Fields. In this case, it is not possible for more than one Virtual Documents activity to use a given RDBMS table. The best solution here is to have a single Virtual Documents activity running on one LEI server in the cluster. For the other servers in the cluster, use full NSFs (in other words, "Not virtual"). This does mean that the connection managed in the Virtual Documents activity is a single point of failure. But, no data would be lost because users could access replica databases on other servers. When the Virtual Documents server comes back online, the changes would then be sent to the RDBMS system.
For more information, see the LEI Activities and User Guide.
LEI now includes SYM files for the Windows 32 platform, which will improve the likelihood of fixes being delivered without first installing debug code. To have this work, we renamed a number of LEI components. These changes should be invisible to users, except in one case: when asked to identify itself, the Lotus Connector for Notes will now report "notesei" instead of "notes."
New LEI 7 features and user interface enhancements
LEI 7 offers a number of new features. For example, before LEI 7, Batch type activities could launch other activities after they would successfully complete. This functionality has been expanded in the Activity Execution section of the Batch type activities (see figure 5).
Figure 5. Activity Execution Options
Now you can designate activities to always run, to only run upon successful completion of the activity, or to run only in the case of a failure. This gives you more control with tasks such as failure cleanup.
The next new LEI 7 feature is also found in the Batch activities. When the Notes connector is in use, an option called Notes password will appear in the Batch activities forms. This option lets you specify that this activity should run with a different Notes ID than the default ID. You don't have to worry about security when your ID has a password on it, because the entire form has Reader/Author privileges that can be set. This feature answers customer requests for improved security when working with Domino data.
The functionality of Virtual Documents activities has been improved. The two features added here deal with operations done in the RDBMS system against virtualized data. These two operations will make the Domino NSF be out of sync with the actual state of the data.
The first operation that used to be able to hurt Virtual Document activities was when a record would be deleted directly in the RDBMS system. Domino would have no idea about this because the delete did not happen "through" Domino. A deleted record would still show up in Domino views, for example. When you tried to open it, you got an error. This problem has been addressed by adding a new thread to the activity that periodically looks for situations such as this to happen. The thread then fixes things up as much as possible, and then brings the Domino views up to a correct state. Any user in the view will see the refresh icon (equivalent to hitting F9). Thereafter, any user who enters the view will see everything as it should be.
In the case of internal keys (where the Virtual Document EI columns are stored directly in the RDBMS data table), the only way to fix the problem is to rebuild the Domino views from scratch. This is because we can't tell what is no longer there, and thus we can't create deletion stubs to make the NSF happy. This is an expensive operation, but at least it can be automated at the activity level. When external keys are in use (EI columns stored in their own table), we are able to find the orphaned record in the external key table and turn it into a deletion stub. Then we can just refresh the Domino views. This is a much more efficient operation. This is turned off by default in the Virtual Documents activity, but can be turned on in the Synchronize Deletions field of the General Options section (see figure 6):
Figure 6. Synchronizing deletions
The other operation that can make data inconsistent is when updates on records occur directly in the RDBMS table. This has the effect of making any columns in the updated records that are also displayed in a Domino view to be out of sync. Also, these changed fields would never replicate. Now in LEI 7, when internal keys are used, LEI can have a thread that looks for this condition. LEI actually looks for an updated time stamp in the EIMODIFIED column. So to get this to work, RDBMS administrators need to make sure that EIMODIFIED gets updated every time a record is updated. This feature is activated via the Synchronize Updates field (see figure 6).
Connection forms enhancements
The connection forms of the LEI Administration database all have a new feature, an action called Test Connectivity. This action (as you probably already figured out) will confirm that your connection properties allow for a successful connection on the LEI server. It basically makes obsolete the need to ever run 'contest' from a command line (see figure 7):
Figure 7. Test Connectivity
When you run the action, you will hopefully get results like those shown in figure 8:
Figure 8. Connection test results
Lotus Connector for ODBC
The Lotus Connector for ODBC form has a new feature. You can now browse for ODBC data sources, as opposed to having to know them on your own. This should help reduce errors when using the ODBC connector, and make things a bit easier. As for ODBC drivers, IBM OEMs drivers from Data Direct. Currently we provide release 5.0 of these drivers. They are available for download from IBM Passport Advantage.
Field mapping in activities has been improved in two different ways in LEI 7. The first way is via the Guess button. If you have many field names to map, and they are similar but not exactly the same between your two connections, then Guess will help you out. You will see the Guess button in the mapping dialog box (see figure 9):
Figure 9. Guess button
In this example, clicking Guess gives you the results shown in figure 10:
Figure 10. Guess results
You can see that the Guess did a decent job of matching up like names and data types. That is definitely easier than doing it yourself. After you are satisfied with the mapping, the results go into the activity document (figure 11):
Figure 11. Mapping Guess results
Note that lines have been added here, to improve the usability of the UI. The lines help you see that your fields are correctly mapped. This is really useful when very long field names (such as those used with SAP) are in use.
All LEI administrator forms are now Sametime-enabled, allowing you to see connection and activity author names, and whether or not those people are active on Sametime.
LEI 7 has a few new reports that help you to administer LEI. They are all found under the Actions Menu\Reports. The first report is called Activity Structure. It basically shows you which connections are used and if any dependent activities are mentioned. Figure 12 shows an example of an Activity Structure report:
Figure 12. Activity Structure report
Each area is linked to the respective documents in LEI via the hotspots shown in blue. This will help you to avoid deleting anything upon which other things rely.
The next report is the Dependent Activity report. It is a lot like the structure report, but does not mention connections used. Figure 13 is a sample Dependent Activity report:
Figure 13. Dependent Activity report
Another very useful new report is the Invalid Link report (see figure 14). Any time that something refers to something else that does not exist, this report will show it to you:
Figure 14. Invalid Link report
The Notes Connection form has been reorganized to make more sense. All of the previous functionality is still present. In addition, all fixed problems that were fixed in the LEI 6.x codestreams as of July 17, 2005 are also included in LEI 7.
Lotus Connector for Oracle
The Oracle connector has been changed and updated. First, the Oracle 7 connector has been retired. (Even Oracle doesn't support Oracle 7 anymore.) The Oracle 8 connector is now the Oracle connector. It has been updated to fully support Oracle 8, 9i, and 10g. Any previously existing connections that were upgraded to LEI 7 that list either the Oracle 7 or Oracle 8 connectors will still continue to function, although you will need to update the connection forms.
This article has covered the major features, enhancements, and changes in LEI 7. As we mentioned, much of our development work went into the support of the Domino cluster. We hope that you find the improved availability of the Domino cluster environment to be as big of a win for LEI as we do. Of course, we think the other additions to LEI 7 are nice to have as well!
- For more information about LEI, see the Lotus Enterprise Integrator product page.
- See the LEI Activities and User Guide for additional LEI information.
- For more information about all the features in Lotus Notes 7.0, see the developerWorks: Lotus article, "New features in Lotus Notes and Domino Designer 7.0."
- And for more information about new features in Lotus Domino 7.0, see the article, "New features in Lotus Domino 7.0."
- See the Lotus Notes/Domino product page for more information about the Lotus Notes and Domino products.