What is new in Lotus Enterprise Integrator (LEI) 7

In this article, we provide a quick tour of new features and enhancements introduced in Lotus Enterprise Integrator (LEI) 7.

Share:

Scott Morris, Senior Software Engineer, IBM

Scott Morris is a Senior Software Engineer and is the Manager of the Lotus Enterprise Integration team of the IBM Software Group. He joined Lotus/IBM in 1990. Since 1997, he has worked on developing products that are offered by the Lotus Enterprise Integration team. He was previously a member of the Notes API team, the Notes server team (before there was a Lotus Domino), and somewhere in the deep dark past, the 1-2-3 for Macintosh team. Scott has an MS in Computer Science from Boston University and a BS in Mathematics from Carnegie Mellon University.



30 August 2005

Also available in Japanese

IBM Lotus Enterprise Integrator for Domino (LEI) allows users to access data across multiple platforms. With LEI, users can work with corporate data, irrespective of where the data actually resides. This lets you leverage your existing store of company knowledge, without having to port this information to a common platform or application. LEI consists of two major components, the LEI Server and the LEI Administrator. The LEI Server monitors the LEI Administrator database for LEI activities to execute. The LEI Administrator (a Notes application) lets users create activities and connections.

IBM recently released Lotus Enterprise Integrator 7, the latest version of LEI. This article gives you a short tour of the enhancements made in LEI 7. These focus on the following two areas:

  • Reliability and Serviceability (RAS) enhancements. In LEI 7, these include support for Domino cluster failover, Batch and RealTime activities, and NSD changes.
  • New features and user interface changes, including Lotus Connector for Oracle improvements.

This article assumes you're familiar with previous versions of LEI. For more information about LEI, see the Lotus Enterprise Integrator product page.

LEI failover

An important RAS change in LEI 7 is support for the Domino cluster. In previous releases, an LEI cluster worked like this: A group of LEI servers could participate in an LEI cluster. You could pick which server in the cluster that an activity would run on by default. If that server went down, another LEI server would pick up that activity and run it. Each server in the LEI cluster used the same LEI Administration database to work with. This last point meant that if the Domino server that hosts the one LEI Administration database went down, then all LEI servers in the cluster went down. In other words, the LEI cluster had a single point of failure.

This is no longer the case. LEI 7 can now be installed on each server in a Domino cluster. Each Domino server will have a cluster replica of both the LEI Administration database and the LEI Log database. So, if any one Domino server goes down, the other LEI servers on the other Domino cluster servers will pick up the work of the inactive server (which they know about in their local replicas).

Setting this up is a breeze. You first find out about the feature in the LEI 7 installer. During the install process, you will see the screen shown in figure 1:

Figure 1. Installing LEI Failover Cluster support screen
Installing LEI Failover Cluster support screen

The installer will either create the first server of the cluster, or join an existing cluster.

In the LEI Administration database, in the Configuration document, we will see the Domino Clustering field enabled and other cluster-related fields properly configured by the installer (see figure 2):

Figure 2. Domino clustering enabled
Domino clustering enabled

As figure 2 shows, below the Domino Clustering field is the Synchronization Delay field. A typical cluster replication is around eight seconds, and then we allow a minute more than that in case system clocks wander a bit. To check the time on a system clock, run the LEI Server Administration\Server Time Check Action from the Actions Menu. You will get a result like that shown in figure 3:

Figure 3. Server Time Check Action result
Server Time Check Action result

If times are too far apart among your servers, you need to go to each server and synchronize them. There are many ways to have the systems stay on Atomic time by themselves. We recommend that you find the correct time synch tool for your operating system. It will make life easier.

The final configuration field is the Batch Activity Failover field (see figure 2). It lets LEI fail over these types of activities (everything except RealTime). The following section explains Batch and RealTime activities.


Batch and RealTime activities

LEI 7 offers two types of activities, Batch and RealTime.

Batch activities

Batch (or Data Management) activities are the only type of activities to actually fail over. This requires that each server in the cluster has identical relational database management system (RDBMS) client configurations (or in the case of the Lotus Connector for SAP, that the SAP Connector be installed on each server in the cluster). The activity author can define the order of failover. For example, always run an activity on server HR1, and if that is down, use HR2. This is done in the Activity Execution Options section of the activity documents (see figure 4):

Figure 4. Activity Execution Options screen
Activity Execution Options screen

After the primary server comes back up, the activity will start running there again.

RealTime activities

The design of LEI RealTime activities requires that they be handled differently in the LEI failover cluster. First, let's look at the Virtual Fields activity. In this case, each LEI server in the Domino cluster must have its own Virtual Fields activity, to run locally. They may all share the same RDBMS table. The Virtual Field activity is smart enough to know that if a record is changed on cluster server A, it does not need to be changed on any other server in the cluster (as long as a Virtual Fields activity is running for that database and form on the other servers). The easiest way to set this up is to create the Virtual Fields activity once, and then copy and paste that activity document once for every server in the cluster. Then edit each activity and set the correct designated server field.

Virtual Documents is significantly different from Virtual Fields. In this case, it is not possible for more than one Virtual Documents activity to use a given RDBMS table. The best solution here is to have a single Virtual Documents activity running on one LEI server in the cluster. For the other servers in the cluster, use full NSFs (in other words, "Not virtual"). This does mean that the connection managed in the Virtual Documents activity is a single point of failure. But, no data would be lost because users could access replica databases on other servers. When the Virtual Documents server comes back online, the changes would then be sent to the RDBMS system.

For more information, see the LEI Activities and User Guide.


NSD changes

LEI now includes SYM files for the Windows 32 platform, which will improve the likelihood of fixes being delivered without first installing debug code. To have this work, we renamed a number of LEI components. These changes should be invisible to users, except in one case: when asked to identify itself, the Lotus Connector for Notes will now report "notesei" instead of "notes."


New LEI 7 features and user interface enhancements

LEI 7 offers a number of new features. For example, before LEI 7, Batch type activities could launch other activities after they would successfully complete. This functionality has been expanded in the Activity Execution section of the Batch type activities (see figure 5).

Figure 5. Activity Execution Options
Activity Execution Options

Now you can designate activities to always run, to only run upon successful completion of the activity, or to run only in the case of a failure. This gives you more control with tasks such as failure cleanup.

The next new LEI 7 feature is also found in the Batch activities. When the Notes connector is in use, an option called Notes password will appear in the Batch activities forms. This option lets you specify that this activity should run with a different Notes ID than the default ID. You don't have to worry about security when your ID has a password on it, because the entire form has Reader/Author privileges that can be set. This feature answers customer requests for improved security when working with Domino data.

Virtual Documents

The functionality of Virtual Documents activities has been improved. The two features added here deal with operations done in the RDBMS system against virtualized data. These two operations will make the Domino NSF be out of sync with the actual state of the data.

The first operation that used to be able to hurt Virtual Document activities was when a record would be deleted directly in the RDBMS system. Domino would have no idea about this because the delete did not happen "through" Domino. A deleted record would still show up in Domino views, for example. When you tried to open it, you got an error. This problem has been addressed by adding a new thread to the activity that periodically looks for situations such as this to happen. The thread then fixes things up as much as possible, and then brings the Domino views up to a correct state. Any user in the view will see the refresh icon (equivalent to hitting F9). Thereafter, any user who enters the view will see everything as it should be.

In the case of internal keys (where the Virtual Document EI columns are stored directly in the RDBMS data table), the only way to fix the problem is to rebuild the Domino views from scratch. This is because we can't tell what is no longer there, and thus we can't create deletion stubs to make the NSF happy. This is an expensive operation, but at least it can be automated at the activity level. When external keys are in use (EI columns stored in their own table), we are able to find the orphaned record in the external key table and turn it into a deletion stub. Then we can just refresh the Domino views. This is a much more efficient operation. This is turned off by default in the Virtual Documents activity, but can be turned on in the Synchronize Deletions field of the General Options section (see figure 6):

Figure 6. Synchronizing deletions
Synchronizing deletions

The other operation that can make data inconsistent is when updates on records occur directly in the RDBMS table. This has the effect of making any columns in the updated records that are also displayed in a Domino view to be out of sync. Also, these changed fields would never replicate. Now in LEI 7, when internal keys are used, LEI can have a thread that looks for this condition. LEI actually looks for an updated time stamp in the EIMODIFIED column. So to get this to work, RDBMS administrators need to make sure that EIMODIFIED gets updated every time a record is updated. This feature is activated via the Synchronize Updates field (see figure 6).

Connection forms enhancements

The connection forms of the LEI Administration database all have a new feature, an action called Test Connectivity. This action (as you probably already figured out) will confirm that your connection properties allow for a successful connection on the LEI server. It basically makes obsolete the need to ever run 'contest' from a command line (see figure 7):

Figure 7. Test Connectivity
Test Connectivity

When you run the action, you will hopefully get results like those shown in figure 8:

Figure 8. Connection test results
Connection test results

Lotus Connector for ODBC

The Lotus Connector for ODBC form has a new feature. You can now browse for ODBC data sources, as opposed to having to know them on your own. This should help reduce errors when using the ODBC connector, and make things a bit easier. As for ODBC drivers, IBM OEMs drivers from Data Direct. Currently we provide release 5.0 of these drivers. They are available for download from IBM Passport Advantage.

Field mapping

Field mapping in activities has been improved in two different ways in LEI 7. The first way is via the Guess button. If you have many field names to map, and they are similar but not exactly the same between your two connections, then Guess will help you out. You will see the Guess button in the mapping dialog box (see figure 9):

Figure 9. Guess button
Guess button

In this example, clicking Guess gives you the results shown in figure 10:

Figure 10. Guess results
Guess results

You can see that the Guess did a decent job of matching up like names and data types. That is definitely easier than doing it yourself. After you are satisfied with the mapping, the results go into the activity document (figure 11):

Figure 11. Mapping Guess results
Mapping Guess results

Note that lines have been added here, to improve the usability of the UI. The lines help you see that your fields are correctly mapped. This is really useful when very long field names (such as those used with SAP) are in use.

Sametime

All LEI administrator forms are now Sametime-enabled, allowing you to see connection and activity author names, and whether or not those people are active on Sametime.

Reports

LEI 7 has a few new reports that help you to administer LEI. They are all found under the Actions Menu\Reports. The first report is called Activity Structure. It basically shows you which connections are used and if any dependent activities are mentioned. Figure 12 shows an example of an Activity Structure report:

Figure 12. Activity Structure report
Activity Structure report

Each area is linked to the respective documents in LEI via the hotspots shown in blue. This will help you to avoid deleting anything upon which other things rely.

The next report is the Dependent Activity report. It is a lot like the structure report, but does not mention connections used. Figure 13 is a sample Dependent Activity report:

Figure 13. Dependent Activity report
Dependent Activity report

Another very useful new report is the Invalid Link report (see figure 14). Any time that something refers to something else that does not exist, this report will show it to you:

Figure 14. Invalid Link report
Invalid Link report

Other changes

The Notes Connection form has been reorganized to make more sense. All of the previous functionality is still present. In addition, all fixed problems that were fixed in the LEI 6.x codestreams as of July 17, 2005 are also included in LEI 7.


Lotus Connector for Oracle

The Oracle connector has been changed and updated. First, the Oracle 7 connector has been retired. (Even Oracle doesn't support Oracle 7 anymore.) The Oracle 8 connector is now the Oracle connector. It has been updated to fully support Oracle 8, 9i, and 10g. Any previously existing connections that were upgraded to LEI 7 that list either the Oracle 7 or Oracle 8 connectors will still continue to function, although you will need to update the connection forms.


Summary

This article has covered the major features, enhancements, and changes in LEI 7. As we mentioned, much of our development work went into the support of the Domino cluster. We hope that you find the improved availability of the Domino cluster environment to be as big of a win for LEI as we do. Of course, we think the other additions to LEI 7 are nice to have as well!

Resources

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into IBM collaboration and social software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Lotus
ArticleID=92909
ArticleTitle=What is new in Lotus Enterprise Integrator (LEI) 7
publish-date=08302005