Distributing query workload by deploying multiple Lifecycle Query Engine servers

By deploying multiple Lifecycle Query Engine instances across a set of servers, you can distribute the query workload and improve performance and scalability. Each Lifecycle Query Engine instance requires its own relational database and contains its own independent triple store index, which indexes the tracked resource set (TRS) data providers that you specify on the Lifecycle Query Engine administration page. Configure each Lifecycle Query Engine instance to index the same set of data providers. Because each Lifecycle Query Engine instance indexes data providers separately, adding additional Lifecycle Query Engine instances puts extra load on your other applications. You should deploy as few Lifecycle Query Engine instances as possible to handle the normal query workload. Consider improving the performance of an existing Lifecycle Query Engine instance by adding additional memory or CPUs instead, because it does not increase the load on the data providers. If you deploy additional Lifecycle Query Engine instances, consider increasing the memory or number of CPUs for the data providers to handle the increased load.

About this task

  • When you upgrade the application, you must stop all the servers and upgrade each one separately. See Migrating Lifecycle Query Engine.
  • Since the different Lifecycle Query Engine servers have their own independent TRS feeds, it is possible that one Lifecycle Query Engine server might have a different set of data, for a certain time period, compared to the other Lifecycle Query Engine server. Load balancing might have a report query run against the first Lifecycle Query Engine server with data set 1 and then later against the second Lifecycle Query Engine server with data set 2. If data set 1 and data set 1 are not the same, the report results are different. Typically, the two data sets are different for a short period of time.

Procedure

  1. Set up the first Lifecycle Query Engine server.
  2. Set up additional servers.
  3. Set up a load balancer.

What to do next

After you set up and configure the Lifecycle Query Engine for load balancing, you can perform administrative tasks on each of the servers. See Administering Lifecycle Query Engine.

Step 1. Set up the first Lifecycle Query Engine server

Procedure

  1. Ensure that the Jazz® Team Server and Lifecycle Query Engine server clocks are synchronized and within 5 minutes of each other.
  2. Install and deploy Lifecycle Query Engine.
  3. Optional: Before starting Lifecycle Query Engine for the first time, in the ../conf/lqe/lqe.node.id file, specify the ID and name for the new installation. If you do not provide this information before starting Lifecycle Query Engine for the first time, a node identifier and name are automatically generated, and the file is created. 
    lqe.node.id=Node1
lqe.node.name=myNewLqeNode
  4. Configure Lifecycle Query Engine authentication, set up data providers, and validate the installation.

    For information about validating Lifecycle Query Engine with the Money that Matters sample, see the Validate Your Lifecycle Query Engine Installation Jazz.net wiki page.

Step 2. Set up additional servers

You will need a backup from your original Lifecycle Query Engine server to configure additional servers. To configure an Lifecycle Query Engine backup schedule, see Backing up and restoring Lifecycle Query Engine.

A backup contains all the type system model in Lifecycle Query Engine and a copy of the indexed data. You can use the backup files to install another Lifecycle Query Engine server that has a copy of the indexed data.

Procedure

  1. Install Lifecycle Query Engine in a new application server. You do not need to install a new Jazz Team Server.
  2. Create a relational database for the new Lifecycle Query Engine server.
  3. Before starting this server for the first time, complete the following steps:
    1. Copy the following folders from the backup to the new installation:
      • Copy the /datasets directory into the conf/lqe directory in the new installation.
      • Copy the /metadata directory into the conf/lqe directory in the new installation.
    2. Copy the lqe.key and lqe.node.id files from the original server to the conf/lqe directory in the new installation.
    3. Create a dbconnection.properties file in the conf/lqe directory of the new installation with the connection info for the new relational database. You can use the dbconnection.properties file from the original installation as a template.
    4. Open the lqe.properties file and set the Lifecycle Query Engine restore property and the config mode property to true: 
      lqe.restore=true
lqe.configMode=true
  4. Start Lifecycle Query Engine.
  5. Go to the Jazz Team Server Setup page: https://fully_qualified_hostname:9443/jts/setup
  6. On the Register Applications page, register the new Lifecycle Query Engine installation:
    1. Click Add Application.
    2. In the Discovery URL field, enter the following value, using the host name and port of the new Lifecycle Query Engine installation: https://fully_qualified_hostname:port/lqe/scr
    3. In the Application Instance field, enter a name that is different from any existing Lifecycle Query Engine installations. The name does not need to match the context root of the installation.
  7. Finish the JTS setup. Do not change any existing values.
  8. Access the new Lifecycle Query Engine installation at https://fully_qualified_hostname:port/lqe/web
    This page will not load until Lifecycle Query Engine has finished restoring from the provided backup, so this step might take some time.
  9. Validate the new installation.

    Open the Lifecycle Query Engine page for the new installation and confirm that the same data providers are configured on this installation and on your original installation. Ensure that the data providers show an up-to-date status and that the last update time for the data providers is after the new Lifecycle Query Engine installation was started.

  10. Optional: Configure a new node name for the new Lifecycle Query Engine installation.
    1. Click the Administration tab.
    2. In the Configuration pane, click LQE Nodes.
    3. Click the link at the top of the right column, showing the current node name.
    4. Enter a new name in the input field and click Save.

Step 3. Set up a load balancer

After you set up the Lifecycle Query Engine nodes, you must set up a load balancer to handle the query workload.

Procedure

  1. Enable workload balancing for your server. See the instructions in the Setting up load balancing for multiple LQEs topic.
  2. Configure Report Builder to contact Lifecycle Query Engine through the load balancer.
    1. To open Report Builder, go to https://fully_qualified_hostname:9443/rs.
    2. In the product banner, click the Administration icon Gear icon for Administration and select Data Sources.
    3. Click the Data Sources tab.
    4. Click the link for the Lifecycle Query Engine data source.
    5. In the SPARQL data source URL field, enter the following URL, using the hostname of the load balancer: https://fully_qualified_hostname:port/lqe/sparql
    6. If you have a Lifecycle Query Engine scoped by a configuration data source, return to the Data Sources page and click the link for that data source.
    7. In the SPARQL data source URL field, enter the following URL, using the hostname of the load balancer: https://fully_qualified_hostname:port/lqe/sparql-variable-config.