Troubleshooting
Problem
During the ELM 6.x to ELM 7.0 on SQL Server 2016 upgrade, DOORS® Next upgrade appears to hang at the "Initialization Complete" phase.
This behavior has only been observed using SQL Server 2016, where we have seen the generation of poor execution plans for key upgrade queries. The problem is that the SQL Server query being run with this plan will take up considerable resources and time to complete, delaying the upgrade progress.
This scenario may continue for several hours, or not complete at all.
Symptom
The symptoms can be detected from various sources.
1) The most obvious symptom is in the console from where the upgrade was initiated. For example:
Setting committed timestamp to 2017-11-11 11:12:13.14 in change set [UUID _a - abCDEFGhiJklm1O_ABC]Setting committed timestamp to 2018-11-11 11:12:13.14 in change set [UUID _a - abCDEFGhiJklm1O_ABC]Setting committed timestamp to 2019-11-11 11:12:13.14 in change set [UUID _a - abCDEFGhiJklm1O_XYZ]Finished setting missing committed timestamps in change sets. Updated 100,000 change sets.Initialization complete.2) The repotools-rm_addTables.log and the repotools_rm_upgrade.log have no new entries for at least two hours.
Depending on the command that was run to initiate the upgrade, the repotools-rm_addTables.log or the repotools_rm_upgrade.log will contain significant logging. During SQL query executions, this will pause for as long as it takes to execute the query.
The addTables.log should reflect point 1 above with:
Initialization complete.
020-07-29 17:10:22,461 Finished setting missing committed timestamps in change sets. Updated 100,000 change sets.
2020-07-29 17:10:22,462 Initialization complete.3) If you are able to see the SQL Server Profiler, then you can see the following query being run. This is also evident in the Resource Monitor for running processes on the Windows server.
SELECT COUNT(*)
FROM RESOURCE.RESOURCE RSC
LEFT JOIN DNGMIGRATION.STORAGE_VERSION sver
ON sver.STORAGE = RSC.URI
LEFT JOIN VVCMODEL.VERSION VRS
ON sver.VERSION_STATE_ID = VRS.STATE_ID
JOIN REPOSITORY.ITEM_STATES STATES
ON RSC.ITEM_ID = STATES.ITEM_UUID
AND NOT EXISTS (SELECT 1 FROM REPOSITORY.ITEM_STATES STATES2
WHERE STATES.ITEM_UUID = STATES2.ITEM_UUID
AND STATES.MODIFIED < STATES2.MODIFIED)
LEFT JOIN REPOSITORY.QUERYABLE_JPI_MAPPING JPI1
ON sver.CONCEPT = JPI1.JPI
LEFT JOIN REPOSITORY.QUERYABLE_JPI_MAPPING JPI2
ON VRS.VERSION = JPI2.JPI
WHERE EXISTS(
SELECT 1 FROM DNGMIGRATION.COMPONENTS_TO_MIGRATE comps
WHERE comps.PROJECT_AREA_ID = RSC.CONTEXT_ID
)
AND RSC.URI like 'storage/com.ibm.rdm.resources/%'
AND NOT EXISTS (SELECT 1 FROM VVCMODEL.VERSION V2
WHERE V2.CONCEPT = VRS.CONCEPT
AND V2.DELETED = 0 -- is not a deletion - these don't have item states
AND VRS.CREATION_TIME < V2.CREATION_TIME)4) It is also it is possible to view this if you generate a javacore. As it is challenging to create a javacore on Windows, we recommend using the following setting in the teamserver.properties prior to upgrade:
com.ibm.service.migration.jvmDumpPeriodMinutes=60
In the javacore file you will be looking for confirmation that this issue has occurred when this query persists for several hours:
3XMTHREADINFO "[SQL@17:10:51.151], SELECT COUNT(*)
FROM RESOURCE.RESOURCE RSC
LEFT JOIN DNGMIGRATION.STORAGE_VERSION sver
...
<the remainder of the query is printed, as stated in point 3 above>
Document Location
Worldwide
[{"Business Unit":{"code":"BU048","label":"IBM Software"},"Product":{"code":"SSUVLZ","label":"IBM Engineering Requirements Management DOORS Next"},"ARM Category":[{"code":"a8m50000000Cj3AAAS","label":"DOORS Next Generation-\u003EDatabase *File under JTS*-\u003ESQL Server"}],"ARM Case Number":"TS003631143","Platform":[{"code":"PF033","label":"Windows"}],"Version":"7.0.0;7.0.1","Line of Business":{"code":"LOB77","label":"Automation Platform"}}]
Log InLog in to view more of this document
This document has the abstract of a technical article that is available to authorized users once you have logged on. Please use Log in button above to access the full document. After log in, if you do not have the right authorization for this document, there will be instructions on what to do next.
Was this topic helpful?
Document Information
Modified date:
01 December 2020
UID
ibm16254393