IBM Support

Initial indexing is slower than expected when LQE/LDX is running on a virtualized Microsoft Windows server

Troubleshooting


Problem

Initial LQE or LDX indexing can be slower than expected (~1 million resources per day) when installed on virtualized Microsoft Windows server environments.

Symptom

Testing shows that LQE/LDX indexing can run at the rate of ~1 million resources per day when running on ideal hardware as described in Best Practices for Configuring LQE for Performance and Scalability.

It has been observed in some cases that initial indexing is much slower.

Cause

LQE/LDX uses Memory Mapped IO mode for accessing the Jena index by default. This is desirable as caching the Jena index in memory results in much faster query performance.

After some investigation, it has been determined that the following code is running slower than expected. The java.nio.MappedByteBuffer.force0 method relies on native operating system methods to flush data to the disk. These methods are typically very fast, however on Microsoft Windows/VMWare have been proven to be a bottleneck.

Thread Name: lqe.BatchWriter[TRS 2.0 for DNG Resources]-thread-0:

at java/nio/MappedByteBuffer.force0(Native Method)
at java/nio/MappedByteBuffer.force(MappedByteBuffer.java:216(Compiled Code))
at com/hp/hpl/jena/tdb/base/file/BlockAccessMapped.flushDirtySegments(BlockAccessMapped.java:247(Compiled Code))
at com/hp/hpl/jena/tdb/base/file/BlockAccessMapped.force(BlockAccessMapped.java:269(Compiled Code))
at com/hp/hpl/jena/tdb/base/file/BlockAccessMapped.sync(BlockAccessMapped.java:138(Compiled Code))
at com/hp/hpl/jena/tdb/base/block/BlockMgrFileAccess.syncForce(BlockMgrFileAccess.java:131(Compiled Code))
at com/hp/hpl/jena/tdb/base/block/BlockMgrWrapper.syncForce(BlockMgrWrapper.java:114(Compiled Code))
at com/hp/hpl/jena/tdb/transaction/JournalControl.replay(JournalControl.java:265(Compiled Code))
at com/hp/hpl/jena/tdb/transaction/JournalControl.replay(JournalControl.java:226(Compiled Code))
at com/hp/hpl/jena/tdb/transaction/TransactionManager.writerCommitsWorker(TransactionManager.java:494(Compiled Code))

Environment

Windows/VMWare

Diagnosing The Problem

You can see what File mode is being used by opening LQE> 'Health Monitoring'> Statistics> (https://<servername>:<port>/lqe/web/health/stats) 'Node Statistics' and 'Tdb File Mode'

Resolving The Problem

A JVM argument is available in LQE 6.0.3 iFix006 and 6.0.4 through Add a hook to force Jena to direct mode (418787) to switch Jena to use Direct-access mode that avoids use of the method found to be the bottleneck.

While enabling this argument should increase the throughput of the initial indexing, it is highly recommended to switch back to memory mapped mode once the initial indexing has completed in order to take advantage of faster query performance.

To enable direct-access mode for Jena, add the JVM argument -Dlqe.jena.direct.mode to the server and set its value to true


For Tomcat or WebSphere Liberty, the argument can be applied by adding the following line in the JVM arguments section in the startup script.

set JAVA_OPTS=%JAVA_OPTS% -Dlqe.jena.direct.mode=true


To set the JVM parameter on WebSphere Application Server, refer to  Technote: Setting generic JVM arguments in WebSphere Application Server

Note: The server must be restarted after making the change in order for the change to take effect.

Once the initial LQE indexing has completed, remove the property and restart the server when possible.

[{"Business Unit":{"code":"BU004","label":"Hybrid Cloud"},"Product":{"code":"SSTU9C","label":"Jazz Reporting Service"},"Component":"Lifecycle Query Engine","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"6.0.1;6.0.2;6.0.3;6.0.4, 6.0.5, 6.0.6, 6.0.6.1","Edition":""}]

Document Information

Modified date:
01 October 2019

UID

swg22007476