Editor's note: Know a lot about this topic? Want to share your expertise? Participate in the IBM Lotus software wiki program today.
Migrating artifacts is a key task when you are migrating your existing Lotus Connections version. Using the migration tool in version 2.0.1, you were able to accomplish this task to some extent; for example, you could migrate some configuration and all data files. You still had to customize many configurations manually, which is one of the most painful tasks in the migration process.
The newly added features in Lotus Connections 2.5 mean that many more configurations must be migrated. The latest migration tool, which is based on XSLT technology, provides a simple and effective way to migrate all necessary configurations and data files. With its help, you don't need to manually reapply all configurations.
This article explains how to use the Lotus Connections 2.5 migration tool, including how to troubleshoot errors and perform a recovery, so that you can understand the migration tool and correctly migrate the artifacts of your existing Lotus Connections environment.
It is assumed that you have a good understanding of IBM WebSphere® Application Server and a general understanding of XSLT and XML Path Language (XPath) technology.
Architecture and work mechanism
Let's begin with an architectural overview of the Lotus Connections V2.5 migration tool. Figure 1 illustrates the high-level, runtime topology of the migration tool.
Figure 1. Runtime topology of the migration tool
The mechanism used for migrating data files is almost the same as that in Lotus Connections 2.0.1, except that the source and target data file lists are different and the mechanism to migrate XML configuration files is completely different. Figure 2 illustrates the work mechanism of the migration tool.
Figure 2. Work mechanism to migrate XML configuration file
A simplified view of the XML configuration files migration is shown in figure 3. The hexagon-shaped elements denote assist files, and the rectangular-shaped elements denote the XML configuration files. You can see the evolution process of the XML configuration files migration.
Figure 3. Simplified view of work mechanism used to migrate XML configuration files
The XML configuration files are located under the directory:
- <WAS_HOME> is the installation location of WebSphere Application Server
- <PROFILE_NAME > is the name of the profile of Lotus Connections
- <CELL_NAME > is the name of the cell, which is a management domain
Most XML configuration files can be customized by users, and some can be extensively customized, so migrating customized configurations is the most important function of the Lotus Connections V2.5 migration tool.
For every XML configuration file to be migrated there is one, and only one, TXT file and zero to three XSL files provided for its migration. An XML configuration file belongs to a given application, and its XSL and TXT files are located under the directory associated with the given application. All such directories are listed here:
Now let's delve into the details of the two phases of the work mechanism to migrate XML configuration files.
Transforming XML configuration files from the source version to the target version
The XML configuration files are in continuous evolution and are tracked by versions. For this article, the source version of the migration is V2.0.1, and the target version is V2.5. Note, however, that there are two “middle versions” between them, namely, V2.5 beta1 and V2.5 beta2.
The migration tool uses XSL files to get the XML configuration files of the target version by applying schema changes on those of the source version. The XSL files are provided with the target version of Lotus Connections.
For some XML configuration files, there is no XSL file because there is no schema change for them between versions 2.0.1 and 2.5. For other XML configuration files, however, there might be multiple XSL files because there can be an XSL file as long as there are schema changes in the next (target) version, including the two middle V2.5 beta1 and V2.5 beta2 versions. In other words, the XSL files are provided only on demand.
The name of the XSL file clearly indicates what it is used for. For example, LotusConnections-config-update-201-25b1.xsl is the XSL file dedicated to the XML file LotusConnections-config.xml. It tells us that there are schema changes for the XML file in V2.5 beta1 compared with V2.0.1, that the version of the source XML configuration file is V2.0.1, and that it is transformed to be V2.5 beta1 by the XSL file.
During the migration process, the V2.0.1 source product is the product in use, but the target V2.5 product must be deployed too. To migrate the XML configuration files, the migration tool exports and stores the V2.0.1 XML configuration files and then handles them on the target product environment.
The target product has XSL files with which the tool transforms the XML configuration files to the target version. The resultant XML configuration files are verified against the corresponding XSD files to ensure that the schema is updated to the target version. After migration, the transformed XML configuration files retain the configurations in the V2.0.1 files.
It not enough, however, to transform their schema only because some environment-specific data of the target system doesn't exist in the transformed XML configuration files. All XML configuration files to be migrated need to go through the next phase.
Replacing fresh-installation XML configuration files with customized configurations in transformed files
Before the migration, the new V2.5 product is deployed as the target environment. Some configuration data in this environment, such as system host and database information, is set in the default XML configuration files by the Lotus Connections 2.5 installer, according to user input during the installation process and information collected from the host environment.
These default V2.5 XML configuration files are hereafter referred to as the fresh-installation XML configuration files. These files don't contain any customized configurations in the source environment, so the migration tool must use the customized configurations from transformed XML configuration files to replace the corresponding configurations in these fresh-installation files.
To do this task, TXT files containing XPath expressions are used to mark the positions of XML configurations to be replaced. For each V2.5 fresh-installation XML file to be replaced, there is one, and only one, relevant TXT file. The migration tool can use the TXT file to do the replacement to get the final migrated XML configuration file.
Let's now discuss the details of the migration procedure.
The migration environment
Although the migration tool can be used in several different processes, such as upgrade, pilot-to-product migration, and product-to-product migration, the use of the migration tool in the different processes is similar. Thus, we use the upgrade from 2.0.1 to 2.5 as our example.
In this detailed walk-through of using the migration tool, the following environments are used:
Lotus Connections 2.0.1 environment
- Operating system: Red Hat Enterprise Linux® ES release 4 (Nahant Update 4)
- WebSphere Application Server V220.127.116.11
Lotus Connections 2.5 environment
- Operating system: Red Hat Enterprise Linux ES release 4 (Nahant Update 7)
- WebSphere Application Server V18.104.22.168
- <WAS_HOME> is the WebSphere Application Server installation home directory.
- <LC_HOME> is the Lotus Connections installation home directory.
- <LC_DATA> is the Lotus Connections data directory.
NOTE: All the commands used are operating system independent.
Using the migration tool to migrate artifacts
The process is separated into three phases.
Exporting Lotus Connections 2.0.1 application artifacts
First, the application artifacts listed here must be exported from Lotus Connections 2.0.
XML configuration files
- <WAS_HOME>/profiles/<PROFILE_NAME>/config/cells/<CELL_NAME>/LotusConnections-config/oa-config.xml is exported to migration/work/export/config/oa-config.xml.
- <WAS_HOME>/profiles/<PROFILE_NAME>/config/cells/<CELL_NAME>/LotusConnections-config/LotusConnections-config.xml is exported to migration/work/export/config/LotusConnections-config.xml.
- <WAS_HOME>/profiles/<PROFILE_NAME>/config/cells/<CELL_NAME>/LotusConnections-config/notification-config.xml is exported to migration/work/export/config/notification-config.xml.
- <WAS_HOME>/profiles/<PROFILE_NAME>/config/cells/<CELL_NAME>/LotusConnections-config/oa-jobs.xml is exported to migration/work/export/config/oa-jobs.xml.
- <WAS_HOME>/profiles/<PROFILE_NAME>/config/cells/<CELL_NAME>/LotusConnections-config/contentreview-config.xml is exported to migration/work/export/config/contentreview-config.xml.
- <WAS_HOME>/profiles/<PROFILE_NAME>/config/cells/<CELL_NAME>/LotusConnections-config/communities-config.xml is exported to migration/work/export/config/communities-config.xml.
- <WAS_HOME>/profiles/<PROFILE_NAME>/config/cells/<CELL_NAME>/LotusConnections-config/forum-config.xml is exported to migration/work/export/config/forum-config.xml.
- <WAS_HOME>/profiles/<PROFILE_NAME>/config/cells/<CELL_NAME>/LotusConnections-config/dogear-config-cell.xml is exported to migration/work/export/config/dogear-config-cell.xml.
- <WAS_HOME>/profiles/<PROFILE_NAME>/config/cells/<CELL_NAME>/LotusConnections-config/profiles-config.xml is exported to migration/work/export/config/profiles-config.xml.
- <WAS_HOME>/profiles/<PROFILE_NAME>/config/cells/<CELL_NAME>/LotusConnections-config/widgets-config.xml is exported to migration/work/export/config/widgets-config.xml.
- <WAS_HOME>/profiles/<PROFILE_NAME>/config/cells/<CELL_NAME>/LotusConnections-config/search-config.xml is exported to migration/work/export/config/search-config.xml.
- <LC_HOME>/activities/activities/activities/activities.properties is exported to migration/work/export/LC_HOME/activities/activities/activitiesactivities.properties.
- <LC_HOME>/blogs/blogs/blogs/blogs.properties is exported to migration/work/export/LC_HOME/blogs/blogs/blogs/blogs.properties.
- <LC_HOME>/communities/communities/communities/communities.properties is exported to migration/work/export/LC_HOME/communities/communities/communities.properties.
- <LC_HOME>/dogear/dogear/dogear/dogear.properties is exported to migration/work/export/LC_HOME/dogear/dogear/dogear.properties.
- <LC_HOME>/profiles/profiles/profiles/profiles.properties is exported to migration/work/export/LC_HOME/profiles/profiles/profiles/profiles.properties.
- <LC_HOME>/homepage/homepage/homepage/homepage.properties is exported to migration/work/export/LC_HOME/homepage/homepage/homepage/homepage.properties.
You can find the other properties in the migration/work/export directory. Actually, the migration tool exports all the properties under <LC_HOME> and then extracts useful properties from it during the importing process.
- <LC_DATA>/activities/content is exported to migration/work/export/data/ACTIVITIES_CONTENT_DIR.
- <LC_DATA>/activities/statistic is exported to migration/work/export/data/ACTIVITIES_STATS_DIR.
- <LC_DATA>/blogs/upload is exported to migration/work/export/data/BLOGS_CONTENT_DIR.
- <LC_DATA>/communities/statistic is exported to migration/work/export/data/COMMUNITIES_STATS_DIR.
- <LC_DATA>/dogear/favorite is exported to migration/work/export/data/DOGEAR_FAVICON_DIR.
- <LC_DATA>/profiles/statistic is exported to migration/work/export/data/PROFILES_STATS_DIR.
You can use the preceding resource lists to validate the export procedure.
To export artifacts from the V2.0.1 environment, perform the following steps:
- Obtain the migration tool from the Lotus Connections 2.5 installation disk.
- Copy the entire Lotus_Connections_Install/migration folder to the V2.0.1 <LC_HOME> directory.
- Open a command line and enter the command:
- Change the directory to the migration folder of the V2.5 migration tool.
- Run the migration lc-export command. The command creates a subdirectory named work under <LC_HOME>/migration/, storing the V2.0.1 application artifacts.
Figure 4 shows an example of the migration directory after export.
Figure 4. Sample directory structure of migration data after export
Import application artifacts on target WebSphere Application Server
After V2.0.1 artifacts are exported, you need to import into the V2.5 target environment those that are useful. Note that not all exported artifacts are imported.
To import V2.0.1 artifacts into the Lotus Connections 2.5 environment, perform these steps:
- Copy the migration folder and exported V2.0.1 artifacts from the V2.0.1 <LC_HOME> directory to the same <LC_HOME> directory in V2.5.
- Open a command line and change to the migration folder.
- Run the command:
Microsoft Windows: migration.bat lc-import
Linux/AIX: ./migration.sh lc-import
Figure 5 shows an example of the migration directory after import.
Figure 5. Sample directory structure of the migration data after import
As the figure shows, the migration/work/import directory contains three temporary folders after import: level10, level11, and level12. The folders contain interim-status files in the migration process, and they can help you diagnose possible problems.
The level10 folder stores the XML configuration files exported from the V2.0.1 source product; level11 stores the fresh-installation V2.5 XML configuration files; level2 stores the interim transformed XML files and the final migrated XML configuration files.
Verify the results in the Lotus Connections 2.5 environment
Follow these steps to verify the migration:
- Check the import logs for any errors. Refer to the “Troubleshooting” section for further information.
- Check the configuration files included in the list described in the “Import application artifact on target WebSphere Application Server” section, making sure that the customized configurations from the V2.0.1 environment are imported successfully.
- Check the data directories described in the “Import application artifacts on target WebSphere Application Server” section, to make sure that the data files under the data directories are imported successfully.
- Verify the applications of the new product, ensuring that they can be started and logged in to successfully, that the web user interface is correct, and that the user configurations have been migrated, for example, the limits of Activities' uploaded files, the handler of Blogs, the Blogs' uploaded images.
Now let's look at some troubleshooting tips.
Using the output messages to troubleshoot
The migration tool outputs the execution tracing in a shell window while the migration is in process. These output messages explain the cause of most problems that might occur with the tool and describe how to fix them. Because the output might be too long to read, we suggest redirecting the output to a log file, using shell output redirection:
migration lc-export > export_output.log
migration lc-import > import_output.log
Verifying the output message
Use the redirected log file containing all output messages to verify the migration result.
First, make sure that all the subtasks have been completed. There are a series of subtasks within the entire migration process, though the commands might look simple. When all the subtasks are completed, the BUILD SUCCESSFUL message displays at the bottom of the log, which looks like this:
Thu Jul 30 22:11:46 CST 2009
[echo] executing post-configuration tasks
Total time: 33 seconds
Return Value: 0
1 file(s) moved.
1 file(s) moved.
1 file(s) moved.
Second, go through the log in detail. The previous step tells you only that all migration tasks are completed, but the result of each task is not clear, so you must go a step further. In this step, review the log to make certain that migration succeeded by searching for “Exception” and “Error” in the log.
If there are no errors in the log, the migration is likely successful. Moreover, if you don't see any harmful information by searching for “Warning” and “Skip”, the migration process is most likely a complete success.
Example case analysis
Now let's use the output message information to troubleshoot an example case in which the export from a V2.0.1 environment fails. If you search for “Exception” in the redirected log file, the error “NullPointerException” shows as highlighted below:
[LoadVersion] Product set = All
[LoadVersion] at com.ibm.lconn.ant.task.LoadProductVersion.execute(Unknown Source)
[LoadVersion] at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:275)
[LoadVersion] at org.apache.tools.ant.Task.perform(Task.java:364)
[LoadVersion] at org.apache.tools.ant.Target.execute(Target.java:341)
[LoadVersion] at org.apache.tools.ant.Target.performTasks(Target.java:369)
[LoadVersion] at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1216)
[LoadVersion] at org.apache.tools.ant.Project.executeTarget(Project.java:1185)
[LoadVersion] at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:40)
[LoadVersion] at org.apache.tools.ant.Project.executeTargets(Project.java:1068)
... [LoadVersion] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[LoadVersion] at java.lang.reflect.Method.invoke(Method.java:615)
[LoadVersion] at com.ibm.wps.config.launch.WpsConfigLauncher.process(WpsConfigLauncher.java:243)
[LoadVersion] at com.ibm.wps.config.launch.WpsConfigLauncher.main(WpsConfigLauncher.java:459)
The error means that the subtask to load the version failed because it was missing something. If you search for “skip” in the log, you get the following, in which the two lines containing “skip” are highlighted:
[LoadLCWasVariable] Server name = server1
[LoadLCWasVariable] Variables of profile AppSrv01 already loaded, skip...
[LoadLCWasVariable] Current feature = search
[LoadLCWasVariable] Feature folder search not installed, skip...
[LoadLCWasVariable] Current feature = mobile
[LoadLCWasVariable] Feature folder mobile not installed, skip...
[LoadLCWasVariable] Current feature = wikis
The preceding trace indicates that the migration tool cannot find the installation folders of the two feature applications Search and Mobile, so the subtask LoadLCWasVariable skipped them.
The Lotus Connections Search component is one of two infrastructure components without which the entire product cannot work. It is impossible that the Search component is not deployed because the existing V2.0.1 environment from which we are exporting is working well.
We need to check the registration information of the feature applications. The registration information of all components is stored in the respective .product files located in <LC_HOME>/version and are named as <component_name>.product.
After checking these .product files, we find that those of Mobile and Search do not exist, which is the cause of the failure. The migration tool cannot find Search and Mobile because their .product files are missing.
In this case, we find both the Exception errors and the Skip warnings in the log, which happen to be caused by the same issue, demonstrating that you might need to group together errors when troubleshooting.
Troubleshooting the most typical problems
The migration command can look simple, but the issues occurring in the process can be complex and difficult to resolve. Here we describe how to troubleshoot the most common problems that you might encounter.
Problem 1. Lack of essential property files. This issue occurs in the export process. Below is the log snippet:
Executing native2ascii with native encoding 'Cp1252': wkplc.properties -> wkplc_ascii.properties
java.lang.Exception: wkplc.properties could not be read.
ERROR: Native2ascii execution failed!
Executing native2ascii with native encoding 'Cp1252': wkplc_comp.properties -> wkplc_comp_ascii.properties
java.lang.Exception: wkplc_comp.properties could not be read.
[06/22/10 22:02:00.521 CST] WKSP0500
C:\Program Files\IBM\WebSphere\LotusConnections\ConfigEngine\config\actions\util s.xml:872: CheckManagedNodeTask: ERROR: invalid input. "WpHome" property cannot be empty or null.
Solution. The exception, “java.lang.Exception: wkplc.properties could not be read” shows twice in the log. The wkplc.properties and wkplc_comp.properties files are configuration files of the back-end engine, located in <LC_HOME>/ConfigEngine/profiles/<PROFILE_NAME>.
These files contain essential configurations, such as the paths of the Lotus Connections product and WebSphere Application Server and the profile paths of different applications. The migration process is also based on the back-end engine. If one or more of these files is missing, or even if some of the configurations are changed to incorrect values by mistake, the migration process fails.
To fix the issue, recover the files or configurations by referring to a working Lotus Connections environment of the same version. Then run the migration tool recovery steps (see the “Recovery from failed import task” section), and rerun the migration command.
Problem 2. Wrong WebSphere Application Server variable value. In this case, the following warning message about the Data file path displays:
Tue Jun 22 22:58:45 CST 2010
[copy] Warning: C:\IBM\WebSphere\LotusConnections\Data\activities\content not found.
Solution. In the V2.5 product, the Data file path of each application is stored in WebSphere Application Server variables, instead of in property files as in V2.0.1. The Data files are placed in a parent directory that is an input parameter of the installation wizard.
You can find the WebSphere Application Server variables in the Environment > WebSphere Variables section of WebSphere Application Server Integrated Solution Console, by default:
where <HOSTNAME> is the host name of the server hosting WebSphere Application Server.
Here the error occurs in the export process. The directory cited in the warning message is activities/content, which belongs to the Activities application. The path information is placed in the WebSphere Application Server variable and, when checking the Integrated Solution Console, you can see the related variable, ACTIVITIES_CONTENT_DIR, and its value, C:\IBM\LotusConnections\Data\activities\content, which is cited in the preceding output message.
The default path is different from this on a Microsoft® Windows® operating system, however, and when we look up the Data directory in the file system directly, the actual directory is C:\Program Files\IBM\LotusConnections\Data\activities\content, rather than the WebSphere Application Server variable value.
This problem might occur when WebSphere Application Server variables are changed manually. To fix the problem, change the WebSphere Application Server variable value to the correct one, and then rerun the migration command.
Problem 3. Error: “Current version is not supported.” If the version of the source environment is not suitable, you see error messages in the export log like this:
[LoadVersion] Load product versions from C:\Program Files\IBM\WebSphere\LotusConnections\ConfigEngine\..\version
[LoadVersion] Prefix =
[LoadVersion] Product set = All
[LoadVersion] Current version is not supported.
[LoadVersion] at com.ibm.lconn.ant.task.LoadProductVersion$Product.fixLCVersion(Unknown Source)
[LoadVersion] at com.ibm.lconn.ant.task.LoadProductVersion$Product.<init>(Unknown Source)
Solution. The migration tool works strictly based on the versions. The V2.5 migration tool supports migration only from V2.0.1. or later / V2.5 to V2.5 and validates the source version at the beginning of the process. In this case, the “Current version is not supported” error means that the version of the source environment is not valid.
To resolve the error, check the version in the .product files of the different applications under the <LC_HOME>/version directory. If the version is lower than V2.0.1, first upgrade it to V2.0.1 with the V2.0.1 migration tool. If the source environment is not deployed with a shipped Gold build, there is no way to support its migration.
Problem 4. Build fails when WebSphere Application Server profiles are in a non-default path. The WebSphere Application Server profile deposits the configurations of WebSphere Application Server applications. The profiles can be deployed under the default path, which is <WAS_HOME>/profiles, and can also be deployed under a non-default path. In this case, the error message is about the non-default path, shown below:
found.[copy]Warning:D:\LC25\WEBSPH~1\APPSER~1\profiles\lc_profile\config\cells\JVConnCell\LotusConnections-config not found.
D:\LC25\WebSphere\LotusConnections\ConfigEngine\config\actions\migrate_util.xml:391: The following error occurred while executing this line:
D:\LC25\WebSphere\LotusConnections\migration\work\import\level2 not found.
Solution. In the log, the path of WebSphere Application Server is D:\LC25\WebSphere\AppServer, whereas the path of the profile deployed applications is D:\LC25\lc_profile, which is the non-default path mentioned above.
The issue is due to the software limitation of V2.5 migration, in addition to other environmental issues. The migration tool doesn't support the non-default profile path, meaning that you cannot place the profile under these paths, or the migration fails.
Recovering from a failed import task
The migration task backs up all files that it needs to modify, making recovery possible.
Recovery is not needed for the export task because it does no harm to the Lotus Connections applications. If there’s an error, follow the troubleshooting steps, and run the export task again.
In contrast, for the import task there is a backup folder in the migration tool after import. The backup directory is <LC_HOME>/migration/work/backup and contains three subdirectories:
- config. Contains all fresh-installation XML configuration files of the target environment. When doing a recovery after a failed import, copy all things in this subdirectory back to <WAS_HOME>/AppServer/profiles/<PROFILE_NAME>/config/cells/<CELL_NAME>/LotusConnections-config recursively.
- Data. Contains all the data files of the target environment before import. When doing a recovery after a failed import, clean <LC_HOME>/data and then copy all things in this subdirectory back recursively.
- LC_HOME. Doesn't need to be recovered because it has not been changed in the migration process.
By leveraging the XSLT and XPath technologies, the Lotus Connections migration tool has shown much improvement in version 2.5. You should now have a good idea of the architecture and mechanism of the new migration tool and how to use it.
By following the troubleshooting tips and the case analysis provided herein you can diagnose and resolve problems more quickly, and by using the recovery instructions you can restore your environment after a failed import.
- Participate in the discussion forum.
- Read the Lotus Connections wiki article, “Using the IBM Lotus Connections 2.0 migration tool to migrate artifacts."
- Refer to the Lotus Connections Information Center.
- Refer to the Lotus Connections wiki.
- Refer to the Lotus Connections product page.
- Refer to the Lotus Connections Support portal.