Backing up and restoring Lifecycle Query Engine and Link Index Provider

You can back up and restore data that is indexed by Lifecycle Query Engine or Link Index Provider. The copy of the indexed data can be used in the future to restore Lifecycle Query Engine or LDX to a previous state.

Note:

The backup and restore feature is available only for Lifecycle Query Engine Jena. So, if you have triple store enabled in Lifecycle Query Engine, then you can backup and restore Lifecycle Query Engine triple store.

Before you begin

Before you can perform backups, make sure that the following requirements are met:
  • You have specified a directory on the server as the backup directory. For more information, see Specifying a backup directory
  • The disk space is at least three times the current size of the indexes under <JTSInstallDir>\server\conf\lqe or <JTSInstallDir>\server\conf\ldx.
  • The server on which you are creating the backup must be mapped or mounted - Uniform Resource Identifiers (URIs) are not supported.
CAUTION:
Use only the built-in backup function to back up Lifecycle Query Engine or LDX.

About this task

You can create a backup of the indexed data, which you can then use to restore the data to a previous state.
Taking the virtual machine(VM) backups for Lifecycle Query Engine or LDX is ineffective and risky for the following reasons:
  • Lifecycle Query Engine and LDX split data between the local storage on the disk and their relational database. If the data in both locations is not coordinated correctly, reports are incorrect and require reindexing as a fix. Running a backup by using the built-in Lifecycle Query Engine or LDX backup option pauses the processing of new changes to ensure that the data that is generated by the backup is consistent. Running VM level backups does not ensure this consistency.
  • The local storage that Lifecycle Query Engine and LDX use is sensitive to being affected by more than one running process. It is safe when only the Lifecycle Query Engine or LDX server is manipulating data. Whereas running any other process while Lifecycle Query Engine or LDX is running, even as simple as reading files, can potentially corrupt the storage. Then, the files must be deleted and all the data must be reindexed from scratch. Using the built-in Lifecycle Query Engine or LDX backup avoids this risk because the required changes are done by the Lifecycle Query Engine or LDX process.

Procedure

  1. On the Lifecycle Query Engine or LDX Administration page, in the navigation pane, click Configuration > Backup.
    Screen capture of the Backup page
  2. You can back up Lifecycle Query Engine or LDX in the following ways:
    • To create backups regularly, in the Scheduled backups section, click Edit.

      On the Backup Schedule window, select Enable Backups to add the following schedule details:

      1. In the Maximum delay to cancel backup field, add the duration for which you want to wait before canceling the scheduled backup. The default value of this property is 10800000 Milliseconds or 3 Hours.
        Note: Update the Maximum delay to cancel backup property only when you want Lifecycle Query Engine to extend the waiting period, due to unreachable server or an ongoing process such as indexing or compaction before canceling the scheduled backup.
      2. Select the run time and days of the week when you want the backup to be created.
      3. Select the Compress backups checkbox if needed.
      4. In the Backup directory on server field, add the path of the backup directory.
    • To create a backup immediately, in the One Time Backup section, click Backup Now, and then type the path of the backup directory. When you request a backup, it starts as all the current activity finishes.
    Note: The backup must be within the window of how long change events are kept from the various data providers. If it is not, you are required to reindex those particular data providers TRS allow LQE to be able to reset the last processed event pointer into the change log of that TRS.
    For DOORS® Next
    The TRS change log is no longer truncated automatically. To enable this capability for the manual incremental rebase, enable Show TRS Change Log truncation options advanced property. From the Engineering Requirements Management DOORS Next admin interface, Debug, check Tracked Resource Set and Truncate Change Log after (days) and specify the number of days to retain (28 by default). Extending this period would only be necessary if a manual incremental rebase is triggered..
    Engineering Workflow Management
    For SCM/RMM Configuration Feed

    The default retention period is 30 days. This can be extended to a longer duration to allow the backup to be valid for a longer period by adjusting the property within the Engineering Workflow Management application under Advanced Property: Engineering Workflow Management TRS Component/com.ibm.team.rtc.trs.service.internal.prune.ChangeLogPruningTask/TRS change log entry retention period in days.

    For Engineering Workflow Management Tracking and Planning
    The default retention period for the work item TRS is a default of 1209600 seconds or 14 days. This can be extended to a longer period to allow the backup to be valid for a longer period by adjusting the property within the Engineering Workflow Management application under Advanced Property: Work Item Component/com.ibm.team.workitem.service.internal.oslc.trs.resources.TrackedResourceSet2ResourceHandler/Tracked Resource Set (TRS) 2.0 retention period.
    Engineering Test Management
    This can be extended to a longer duration to allow the backup to be valid for a longer period by adjusting the property within the Engineering Workflow Management application Maximum Age (days): Maximum number of days to keep TRS change events before the background task deletes them from the repository. If the value is 0, the task deletes all TRS change events. If the value is less than 0, the task does not delete any TRS change events. (Default: 28 days)

Restoring Lifecycle Query Engine or Link Index Provider from a backup directory

Based on the number of backups, one or more time-stamped directories exist within the backup directory. Each time-stamped directory contains the following directories:
  • datasets: This directory contains one subdirectory for each partition in the index. Each of the subdirectory contains the following content:
    • <partition-name>\config_manifest.json: Manifest file that indicates where the indexes for the partition must exist on the disk.
    • <partition-name>\indexTdb: Backup of the Lifecycle Query Engine or LDX index database for the partition.
    • <partition-name>\textIndex: Backup of the Lifecycle Query Engine or LDX Lucene index for the partition. This index enables full-text search of the indexed data.

      The creation of the textIndex is configurable. In the Lifecycle Query Engine Advanced Properties page, if you select the Create Index for Full Text Search option, then the default\textIndex subdirectory is created. Otherwise, it is not created.

    Note:
    The following partitions appear in Lifecycle Query Engine and LDX backups:
    • default: Contains the backup of the Artifact (primary) partition, which holds the artifacts and relationships that are created by Engineering Lifecycle Management tools.
    • version: Contains the backup of the internal type system model partition, which holds type system model that is used to optimize reindexing and validation.
    The following partitions appear in Lifecycle Query Engine backups:
    • shapes: Contains the backup of the Artifact Type Data partition. It holds the resource shape information that defines the properties and links, which are available on the specific artifact types.
    • history: Contains the backup of the Historical Data partition, which holds the historical metric data that is used to create trend reports.
  • metadata: Contains backup properties and settings. It contains files that represent all the data in the Lifecycle Query Engine relational database.

Before you begin

Before you restore the backup, check the configuration in the data providers and see whether backup is in the range of that configuration. If the backup is too old and not within the configuration of data providers, the back up is not useful, producing a change log truncation error. As a result, you need to reindex in LQE.

Procedure

  1. Shut down the servlet engine that contains Lifecycle Query Engine or LDX.
  2. Rename the current indexes.
    • For Lifecycle Query Engine rename any of the following directories that exist in <JTSInstallDir>\server\conf\lqe:
      • indexTdb
      • textIndex
      • shapeTdb
      • shapeText
      • historyTdb
      • historyText
      • versionTdb
    • For LDX, rename any of the following directories that exist in <JTSInstallDir>\server\conf\ldx:
      • indexTdb
      • textIndex
      • versionTdb
  3. Copy the backed-up Lifecycle Query Engine indexes to the server.
    1. Copy the backed-up directory Backup<timestamp>\datasets into <JTSInstallDir>\server\conf\lqe, or <JTSInstallDir>\server\conf\ldx.
    2. Copy the backed-directory Backup<timestamp>\metadata into <JTSInstallDir>\server\conf\lqe or <JTSInstallDir>\server\conf\ldx.
  4. Edit the lqe.properties file in <JTSInstallDir>\server\conf\lqe\lqe.properties or <JTSInstallDir>\server\conf\ldx\ldx.properties. Set the property lqe.restore=true.
  5. Start the servlet engine that contains Lifecycle Query Engine or LDX.