Troubleshooting
Problem
After upgrading to ITNCM 6.4.1.1, some worker servers will not start up.
Symptom
The startup script hangs for a long time and then eventually returns a "Not Running" for the server status. In addition, the servers window in the ITNCM GUI will have a blank under the "drivers checksum" column.
The customer will also receive various JPA/DB errors in the ITNCM server logs. Below are some example errors the customer may see in various logs:
Intelliden.log
---------------
2015/04/26,08:34:02.113,com.ibm.tivoli.itncm.logging.ItncmLogger,THR:213,ERROR,Exception while deleting locks for finished tasks
java.lang.IllegalArgumentException: Parameter "Parameter<Timestamp>(2)" declared in "SELECT DISTINCT(lk) FROM DeviceLock lk, AuditLog al WHERE lk.lockable.uniquekey=al.devicekey AND
lk.task.uniquekey IN (SELECT tk.uniquekey FROM Task tk WHERE tk.uow.uniquekey=al.uowkey AND tk.status=?1) AND al.createdtime<?2 AND al.createdtime= (SELECT MAX(al1.createdtime) FROM AuditLog al1 WHERE al1.devicekey=al.devicekey AND al1.uowkey=al.uowkey)" is set to value of "4/26/15 5:34 AM" of type "java.util.Date", but this parameter is bound to a field of type "java.sql.Timestamp".
at org.apache.openjpa.persistence.QueryImpl.assertValueAssignable(QueryImpl.java:1085)
at org.apache.openjpa.persistence.QueryImpl.bindValue(QueryImpl.java:1042)
... 4 more
2015/04/26,08:22:14.926,com.intelliden.core.workflow.scheduler.OperationScheduler,THR:1,ERROR,Failed when attempting to clear redundant locks
<openjpa-2.0.1-r422266:989424 fatal user error> org.apache.openjpa.persistence.ArgumentException: There is no query with the name "selectExecutingUows" defined for any of the known persistent classes: [com.ibm.tivoli.ncm.jpa.objects.OsRegistry,
com.ibm.tivoli.ncm.jpa.objects.PolicyExemptionByVtmos, com.ibm.tivoli.ncm.jpa.objects.PolicyExemptionByDevice,
com.ibm.tivoli.ncm.jpa.objects.EventPoller, com.ibm.tivoli.ncm.jpa.objects.AlarmAction,
com.ibm.tivoli.ncm.jpa.objects.RestoredPreCompliance, com.ibm.tivoli.ncm.jpa.objects.specific.TaskResultInformation,
... 4 more
Server.err
------------
Exception in thread "main" java.lang.ExceptionInInitializerError
at java.lang.J9VMInternals.initialize(J9VMInternals.java:223)
at com.intelliden.core.workflow.admin.WorkflowComponent.start(WorkflowComponent.java:110)
at com.intelliden.component.internal.ComponentManager.startComponents(ComponentManager.java:360)
at com.intelliden.component.internal.ComponentManager.start(ComponentManager.java:160)
at com.intelliden.core.admin.StartStop.main(StartStop.java:9)
Caused by: <openjpa-2.0.1-r422266:989424 fatal user error> org.apache.openjpa.persistence.ArgumentException: There is no
query with the name "selectAllDriversOrderByVersion" defined for any of the known persistent classes:
[com.ibm.tivoli.ncm.jpa.objects.OsRegistry, com.ibm.tivoli.ncm.jpa.objects.PolicyExemptionByVtmos,
com.ibm.tivoli.ncm.jpa.objects.PolicyExemptionByDevice, com.ibm.tivoli.ncm.jpa.objects.EventPoller,
... 4 more
Cause
The installer should automatically create a symbolic link for the DB jpa jar file. Occasionally, this symbolic link does not get created, thus ITNCM cannot properly communicate with the DB.
Resolving The Problem
An immediate workaround is to manually create a symbolic link using the procedure below:
If you are using Oracle for your database -
1) Navigate to the /opt/IBM/tivoli/netcool/ncm/lib folder.
2) type "ln -s ibm_tivoli-jpa_oracle.jar ibm_tivoli-jpa.jar" (without the quotes) and press "enter".
3) If successful, an LS listing should display the following -
ibm_tivoli-jpa.jar -> ibm_tivoli-jpa_oracle.jar
If you are using DB2 for your database -
1) Navigate to the /opt/IBM/tivoli/netcool/ncm/lib folder.
2) type "ln -s ibm_tivoli-jpa_db2.jar ibm_tivoli-jpa.jar" (without the quotes) and press "enter".
3) If successful, an LS listing should display the following -
ibm_tivoli-jpa.jar -> ibm_tivoli-jpa_db2.jar
Once the link has been created, start ITNCM.
L3 is investigating why the symbolic link is not getting created all of the time and will implement a fix if possible.
Was this topic helpful?
Document Information
Modified date:
17 June 2018
UID
swg21883942