Reducing downtime during Lifecycle Query Engine reindexing

Edit online
Link Index ProviderWhen you reindex Lifecycle Query Engine (LQE), reports that run during the reindexing might not be accurate. Team members must run reports only after the reindexing is complete. To reduce this downtime while you reindex LQE, you can clone the LQE instance and reindex the clone. Team members can continue to run reports and historical trend data is still collected. After the clone is reindexed, you can merge the indexed data and resume by using the original LQE instance.

About this task

In the following steps, LQE1 refers to the original LQE instance and LQE2 refers to the clone of LQE1.

Procedure

  1. Set up the first LQE instance named LQE1.
  2. Clone the LQE1 instance to create another instance named LQE2.
  3. Reindex the LQE2 instance.
  4. Merge the indexed data from the LQE2 instance into the LQE1 instance.
  5. Restart the LQE1 instance and configure the database connection.

1. Setting up the first LQE instance named LQE1

Edit online

About this task

If your system already has an LQE instance, go to 2. Cloning the LQE1 instance to create another instance named LQE2.

Procedure

  1. Ensure that the Jazz Team Server and Lifecycle Query Engine server clocks are synchronized and within 5 minutes of each other.
  2. Install and deploy LQE.
  3. Before you start LQE for the first time, complete the following steps:
    1. Configure an external database. See Supported Database Setup information on the Jazz.net wiki.
    2. Optional: In the ../conf/lqe/lqe.node.id file, specify the ID and name for the new installation. If you do not provide this information before you start LQE the first time, a node identifier and name are automatically generated, and the file is created. 
      lqe.node.id=Node1
lqe.node.name=myNewLqeNode
  4. Configure LQE authentication, set up data providers, and validate the installation.

    For information about validating LQE with the Money that Matters sample, see Validate Your Lifecycle Query Engine Installation on the Jazz.net wiki.

2. Cloning the LQE1 instance to create another instance named LQE2

Edit online

Generate a cloned instance named LQE2 from a backup of the original LQE instance (LQE1). To configure an LQE backup schedule, see Backing up and restoring Lifecycle Query Engine and Link Index Provider.

Note: Context root of LQE2 must be same as LQE 1.

A backup contains all the type system model in LQE and a copy of the indexed data.

Procedure

  1. Run an LQE backup on the original LQE instance (LQE1).
  2. Install LQE in a new servlet engine. You do not need to configure a new Jazz Team Server or application server.
  3. Create a relational database for the new LQE instance (LQE2).
  4. Before you start the LQE2 instance for the first time, complete the following steps:
    1. Copy the following folders from the LQE1 instance to the LQE2 instance:
      • Copy /datasets into conf/lqe
      • Copy /metadata into conf/lqe
    2. Copy the lqe.key and lqe.node.id files from the LQE1 instance into the conf/lqe folder in the LQE2 instance.
    3. In the /server/conf/lqe/lqe.properties file of the LQE2 instance, set this LQE property: lqe.restore=true
    4. Optional: Specify a name for the new installation, such as LQE2. In the lqe.node.id file in the new installation, edit the lqe.node.name property to specify the new name. Do not change the lqe.node.id property.
  5. Start the LQE2 instance and configure the database connection.

    It might take a while for the LQE2 instance to start.

    1. Go to the JTS Setup page at https://[fully qualified hostname]:9443/jts/setup
    2. Select Custom Setup and advance through the pages until you reach the Register Applications page. Do not change any of the existing values.
    3. On the Register Applications page, register the new LQE installation.
      1. Click Add Application.
      2. In the Discovery URL field, enter the https://[fully qualified hostname]:[port]/lqe/scr value, using the hostname and port of the new LQE installation.
      3. In the Application Instance field, enter a name that is different from any of the existing LQE installations. The name does not need to match the context root of the installation.
    4. Finish JTS setup. Do not change any of the existing values.
    5. Go to https://host:port/lqe. A configuration page is shown. Enter the database connection information for a new, empty LQE database. Do not use the same database as the LQE1 instance.
    6. Save your changes.
    The LQE home page is shown, which lists the data providers for the LQE2 instance.
  6. Validate the LQE2 instance. Go to the LQE page for the LQE2 instance and verify that the same data providers are configured for both instances. Verify that the data providers update successfully.

Results

You now have two LQE instances that contain the same-indexed data and historical trend data. You can now reindex the LQE2 instance. The LQE1 instance handles reporting requests and continues to collect historical trend data.

3. Reindexing the LQE2 instance

Edit online
To reindex the LQE2 instance, take the following steps:

About this task

For more information about the reindexing operation, see Managing data providers for Lifecycle Query Engine.

Procedure

  1. Open the LQE Data Providers page at https://host:port/lqe/web/admin/data-sources.
  2. Click Reindex All. Depending on the amount of data, creating the initial index or reindexing the data providers can take a long time.
    Note: When indexing or reindexing is in progress, do not shut down or restart the server. If the server shuts down or restarts, you need to restart the indexing again.

4. Merging the indexed data from the LQE2 instance into the LQE1 instance

Edit online
To ensure that the LQE1 instance contains a complete and accurate data set for reporting, merge the indexed data from the LQE2 instance with the current data on the LQE1 instance. These steps assume that you continue to use the LQE1 instance as the LQE server, and not the LQE2 clone.

Procedure

  1. Back up both the LQE1 and LQE2 instances. For details, see Backing up and restoring Lifecycle Query Engine and Link Index Provider.

    Backing up the LQE1 instance ensures that you preserve the history in case it is deleted accidentally.

    Backing up the LQE2 instance provides you with the data you need to merge both LQE instances.

  2. Stop both LQE instances.
  3. In the lqe.properties file of the LQE1 instance, set this LQE property: lqe.restore=true.
  4. Overwrite the history of the LQE1 instance (not the new LQE1 backup).
    1. Copy the following folders from the LQE2 backup to the LQE1 instance:
      • Copy /datasets into conf/lqe

      • Copy /metadata into conf/lqe

    2. In the LQE1 instance, delete the following folders, which are subfolders of what you copied.
      • conf/lqe/datasets/history/indexTdb
      • conf/lqe/datasets/history/textIndex
    3. Copy the LQE1 history folders to the data sets folders.
      • Copy conf/lqe/historyTdb to conf/lqe/datasets/history/indexTdb
      • Copy conf/lqe/historyText to conf/lqe/datasets/history/textIndex
    The LQE1 instance now contains fully indexed, up-to-date data. Your work with the LQE2 instance is complete.

5. Restarting the LQE1 instance and configuring the database connection

Edit online

Procedure

Restart the LQE1 instance. The CLM applications in this deployment resume their regular use of the LQE1 instance.
Note: If you are prompted to register the LQE1 instance with Jazz Team Server, complete these steps:
  1. In the lqe.properties file, set this LQE property: configMode=true.
  2. Register the LQE1 instance again.
  3. Restart the LQE1 instance.

You do not need to start the LQE2 instance.