It has been observed in some cases that initial indexing is much slower.
After some investigation, it has been determined that the following code is running slower than expected. The java.nio.MappedByteBuffer.force0 method relies on native operating system methods to flush data to the disk. These methods are typically very fast, however on Microsoft Windows/VMWare have been proven to be a bottleneck.
Thread Name: lqe.BatchWriter[TRS 2.0 for DNG Resources]-thread-0:
at java/nio/MappedByteBuffer.force0(Native Method)
at java/nio/MappedByteBuffer.force(MappedByteBuffer.java:216(Compiled Code))
at com/hp/hpl/jena/tdb/base/file/BlockAccessMapped.flushDirtySegments(BlockAccessMapped.java:247(Compiled Code))
at com/hp/hpl/jena/tdb/base/file/BlockAccessMapped.force(BlockAccessMapped.java:269(Compiled Code))
at com/hp/hpl/jena/tdb/base/file/BlockAccessMapped.sync(BlockAccessMapped.java:138(Compiled Code))
at com/hp/hpl/jena/tdb/base/block/BlockMgrFileAccess.syncForce(BlockMgrFileAccess.java:131(Compiled Code))
at com/hp/hpl/jena/tdb/base/block/BlockMgrWrapper.syncForce(BlockMgrWrapper.java:114(Compiled Code))
at com/hp/hpl/jena/tdb/transaction/JournalControl.replay(JournalControl.java:265(Compiled Code))
at com/hp/hpl/jena/tdb/transaction/JournalControl.replay(JournalControl.java:226(Compiled Code))
at com/hp/hpl/jena/tdb/transaction/TransactionManager.writerCommitsWorker(TransactionManager.java:494(Compiled Code))
Diagnosing The Problem
Resolving The Problem
While enabling this argument should increase the throughput of the initial indexing, it is highly recommended to switch back to memory mapped mode once the initial indexing has completed in order to take advantage of faster query performance.
To enable direct-access mode for Jena, add the JVM argument
-Dlqe.jena.direct.mode to the server and set its value to true
For Tomcat or WebSphere Liberty, the argument can be applied by adding the following line in the JVM arguments section in the startup script.
set JAVA_OPTS=%JAVA_OPTS% -Dlqe.jena.direct.mode=true
To set the JVM parameter on WebSphere Application Server, refer to Technote: Setting generic JVM arguments in WebSphere Application Server
Note: The server must be restarted after making the change in order for the change to take effect.
Once the initial LQE indexing has completed, remove the property and restart the server when possible.
01 October 2019