Distributing query workload by deploying multiple Lifecycle Query Engine servers

Edit online

By deploying multiple Lifecycle Query Engine (LQE) instances across a set of servers, you can distribute the query workload and improve performance and scalability. Each LQE instance requires its own relational database and contains its own independent triple store index, which indexes the tracked resource set (TRS) data providers that you specify on the LQE administration page. Configure each LQE instance to index the same set of data sources. Because each LQE instance indexes data sources separately, adding additional LQE instances puts extra load on your other applications. You should deploy as few LQE instances as possible to handle the normal query workload. Consider improving the performance of an existing LQE instance by adding additional memory or CPUs instead, because it does not increase the load on the data sources. If you deploy additional LQE instances, consider increasing the memory or number of CPUs for the data sources to handle the increased load.

About this task

If you are deploying LQE on WebSphere® Application Server, you must update the application server configuration to include a few custom properties. See Step 4. Optional: Configure WebSphere Application Server for load balancing.
When you upgrade the application, you must stop all the servers and upgrade each one separately. See Migrating Lifecycle Query Engine.
Since the different LQE servers have their own independent TRS feeds, it is possible that one LQE server might have a different set of data, for a certain time period, compared to the other LQE server. Load balancing might have a report query run against the first LQE server with data set 1 and then later against the second LQE server with data set 2. If data set 1 and data set 1 are not the same, the report results are different. Typically, the two data sets are different for a short period of time.

Procedure

Set up the first LQE server.
Set up additional servers.
Set up a load balancer.
Optional: Configure WebSphere Application Server for load balancing.

What to do next

After you set up and configure the LQE for load balancing, you can perform administrative tasks on each of the servers. See Administering Lifecycle Query Engine.

Step 1. Set up the first LQE server

Edit online

Procedure

Ensure that the Jazz® Team Server and Lifecycle Query Engine server clocks are synchronized and within 5 minutes of each other.
Install and deploy LQE.
Optional: Before starting LQE for the first time, in the ../conf/lqe/lqe.node.id file, specify the ID and name for the new installation. If you do not provide this information before starting LQE for the first time, a node identifier and name are automatically generated, and the file is created.
```
lqe.node.id=Node1
lqe.node.name=myNewLqeNode
```
Configure LQE authentication, set up data sources, and validate the installation.

For information about validating LQE with the Money that Matters sample, see the Validate Your Lifecycle Query Engine Installation Jazz.net wiki page.

Step 2. Set up additional servers

Edit online

You will need a backup from your original LQE server to configure additional servers. To configure an LQE backup schedule, see Backing up and restoring Lifecycle Query Engine.

A backup contains all the metadata in LQE and a copy of the indexed data. You can use the backup files to install another LQE server that has a copy of the indexed data.

Procedure

Run an LQE backup on the original server.
Install LQE in a new application server. You do not need to install a new Jazz Team Server.
Create a relational database for the new LQE server.
Before starting this server for the first time, complete the following steps:
1. Copy the following folders from the backup to the new installation:
  - Copy the /datasets directory into the conf/lqe directory in the new installation.
  - Copy the /metadata directory into the conf/lqe directory in the new installation.
2. Copy the lqe.key and lqe.node.id files from the original server to the conf/lqe directory in the new installation.
3. Create a dbconnection.properties file in the conf/lqe directory of the new installation with the connection info for the new relational database. You can use the dbconnection.properties file from the original installation as a template.
4. Open the lqe.properties file and set the LQE restore property and the config mode propertty to true:
```
lqe.restore=true
lqe.configMode=true
```
Start LQE.
Go to the Jazz Team Server Setup page: https://fully_qualified_hostname:9443/jts/setup
On the Register Applications page, register the new LQE installation:
1. Click Add Application.
2. In the Discovery URL field, enter the following value, using the host name and port of the new LQE installation: https://fully_qualified_hostname:port/lqe/scr
3. In the Application Instance field, enter a name that is different from any existing LQE installations. The name does not need to match the context root of the installation.
Finish the JTS setup. Do not change any existing values.
Access the new LQE installation at https://fully_qualified_hostname:port/lqe/web
This page will not load until LQE has finished restoring from the provided backup, so this step might take some time.
Validate the new installation.

Open the LQE page for the new installation and confirm that the same data sources are configured on this installation and on your original installation. Ensure that the data sources show an up-to-date status and that the last update time for the data sources is after the new LQE installation was started.
Optional: Configure a new node name for the new LQE installation.
1. Click the Administration tab.
2. In the Configuration pane, click LQE Nodes.
3. Click the link at the top of the right column, showing the current node name.
4. Enter a new name in the input field and click Save.

Step 3. Set up a load balancer

Edit online

After you set up the Lifecycle Query Engine nodes, you must set up a load balancer to handle the query workload.

Procedure

Enable workload balancing for your server. See the instructions in the Setting up load balancing for multiple LQEs topic. You can also see the same instructions in the Configure IBM HTTP Server for load balancing Jazz.net wiki page.
Configure Report Builder to contact LQE through the load balancer.
1. To open Report Builder, go to https://fully_qualified_hostname:9443/rs.
2. In the product banner, click the Administration icon and select Data Sources.
3. Click the Data Sources tab.
4. Click the link for the Lifecycle Query Engine data source.
5. In the SPARQL data source URL field, enter the following URL, using the hostname of the load balancer: https://fully_qualified_hostname:port/lqe/sparql
6. If you have a Lifecycle Query Engine scoped by a configuration data source, return to the Data Sources page and click the link for that data source.
7. In the SPARQL data source URL field, enter the following URL, using the hostname of the load balancer: https://fully_qualified_hostname:port/lqe/sparql-variable-config.

Step 4. Optional: Configure WebSphere Application Server for load balancing

Edit online

If you use Lifecycle Query Engine with WebSphere Application Server, you must set three custom properties for the application server. These custom properties are required to pass host port information through the load balancer proxy. By default, WebSphere Application Server reads the port number from the request URL instead of from the HTTP header. These properties override the default.

Procedure

In the WebSphere Application Server administrative console, go to Servers > Server Types, and click WebSphere application servers.
On the Application servers page, click Server1.
On the Configuration page, click Web Container Settings.
Click Custom properties and specify the following values:
- com.ibm.ws.webcontainer.extractHostHeaderPort = true
- trusted = false
- trusthostheaderport = true