Validating TRS feeds and the LQE or Link Index Provider index

Edit online
Validate all the tracked resource set (TRS) feeds registered in the Lifecycle Query Engine (LQE) or LDX index when Report Builder reports don’t show the artifacts that you expect. Validating these items can help you diagnose and troubleshoot problems and avoid reindexing data providers.

About this task

You may validate only TRS feeds, for which, validation is supported by applications, in these situations:
  • A report is missing artifacts.
  • A report includes outdated (stale) artifacts.
  • A report shows the same artifact more than once, possibly with different attribute values or with its old and new name.
  • An application view and a corresponding report in Jazz® Reporting Service (JRS) show a different number of artifacts.
  • A report includes deleted artifacts.
  • LQE or LDX shows skipped resources.
    Tip: To validate that you do not have skipped resources, make sure Validate artifacts option is selected.
Validating all the TRS feeds that are registered in the LQE or LDX index can increase your confidence in the quality of your reports and help you troubleshoot missing, stale, or extra artifacts:
  • TRS Validation works by itself, without accessing LQE or LDX. It compares only the actual and expected content in TRS feed.
Note:
  • Do not run more than one TRS validation operation at a time for a given TRS.
  • If the application server is stopped during a TRS Validation operation, then, when the server is restarted, the TRS validation is not automatically restarted and it is in an unknown state. The admin user must set the option to clear the cache when starting a new validation.
The steps to validate TRS feeds are different across Engineering Lifecycle Management applications. Complete the steps in this topic for the application whose feed you want to validate.
To learn more about validation before you do it, see the following resources:

For a description of TRS feeds in Engineering Lifecycle Management, see Adding Tracked Resource Set feeds for lifecycle management applications.

Jazz Team Server, Engineering Workflow Management, Rhapsody Model Manager, Engineering Requirements Management DOORS Next, Engineering Test Management, and Global Configuration Management

Edit online
Validate TRS feeds on the TRS Feed Validation page in the administration section of the application.

About this task

The following are some examples of TRS feeds:
  • JTS User Resources (TRS 2.0): This feed contains information about Jazz Team Server users. If a report about system users isn't showing the expected results, and you confirm that 10 users are registered in the repository, the validation task checks whether the TRS feed has 10 distinct references to users.
  • Engineering Workflow Management Process Resources (TRS 2.0): If a report doesn’t show the project areas, team areas, or iterations that you expect, the validation task can help you determine whether they are missing from this TRS feed. If a user runs a report that doesn't include artifacts from a project that the user belongs to, TRS feed validation can fix the permission problem. After the validation, the report shows all the expected data.
  • GCM Resources (TRS 2.0): This feed contains information about changes to global configurations, for example, when they were created, updated, or deleted.
Tip: It is recommended to run validation when the server is not under load. Maintenance windows, when nobody makes changes on the server are preferable. Validation is potentially heavy operations, consuming a lot of memory and processor time.

Procedure

  1. Open the application administration page for the TRS feed to validate:
    • Jazz Team Server: https://host:port/jts/admin
    • Engineering Workflow Management or Rhapsody® Model Manager: https://host:port/ccm/admin
    • Engineering Requirements Management DOORS® Next: https://host:port/rm/admin
    • Engineering Test Management: https://host:port/qm/admin
    • Global Configuration Management (GCM): https://host:port/gc/admin
  2. On the toolbar, click TRS Feed Diagnostics.
  3. In the Validate dialog box, click Validate artifacts and configurations. The TRS is read and validated.
    For Engineering Test Management you can also go to https://host:port/qm/TRSValidator and complete the following steps.
    1. For a full comparison of the TRS feed artifacts with those in the repository, click compare > runCompare. When the comparison starts, you can check the progress or stop the process.
    2. To see the log of all validations that ran on the system or any validation that is still in progress, click trsLogs.
    3. When the comparison ends, check the logs for messages, such as Items match exactly or Discrepancies were found.
    4. If discrepancies are found, they are automatically fixed if you set the Automatically resolve discrepancies option for the com.ibm.rqm.oslc.trs.validator.TRSValidationWatchdog property to true.

      If you did not change the advanced property, you can start cleanup events at https://host:port/qm/TRSValidator by clicking compare > cleanDifferences.

    Note:

    You can use an alternative Engineering Test Management validation tool to automatically fix any discrepancies. The tool requires an advanced property before you start the validation process.

    To set the property, go to the administration page at https://host:port/qm/admin and find the Advanced Properties section. Then, change the Automatically resolve discrepancies option for the com.ibm.rqm.oslc.trs.validator.TRSValidationWatchdog property to true.

    Regarding DOORS Next TRS Validation:

    For running DOORS Next TRS Validation, it requires the Selections Validation disk cache directory advanced property to be set prior to validation.

    Note: To run TRS Validation for DOORS Next resources, the Selections Validation disk cache directory advanced property must be set, pointing to a directory on the application server. This directory will contain configuration Selections cache data, which may be large. If there is a need to relocate the cache directory, it can be done by changing the property value. After changing the property value, selecting the Clear cached data option is required during the next validation.

What to do next

If the problem persists, ask an LQE or LDX administrator to validate the LQE or LDX index.

Lifecycle Query Engine or Link Index Provider

Edit online
Validate the LQE or LDX index to compare it with a TRS feed from an Engineering Lifecycle Management application. This task finds missing, stale (outdated), or extra artifacts.

Before you begin

  • If you upgrade from 6.0.6 or an earlier version, a migration task creates a new partition that is used for validation and shortens reindexing time. This process might take some time.
  • If you upgrade from any earlier version, add the following properties to the JazzInstallDir/server/conf/log4j2.xml file to ensure that your validation logs use the latest format. 
    # Validation Task
<Configuration name="ApplicationLogging" status="error" monitorInterval="10">
    <Properties>
        <Property name="pattern">%d{ISO8601} [%30t] %5p %-50.50c %X{Tid} - %m%n</Property>
        <Property name="ansipattern">%d{ISO8601} [%-5.5t] %5p %c{1} %X{Tid} %highlight{%m}%n</Property>
        <Property name="app">lqe</Property>
        <Property name="dir">logs</Property>
    </Properties>
	<Appenders>
	<!-- Validation task log file -->
		<RollingFile name="lqeValidationTaskLog" filename="${dir}/${app}-validation.log" filePattern="${dir}/${app}-validation-%i.log">
			<PatternLayout pattern="${pattern}"/>
			<Policies>
				<SizeBasedTriggeringPolicy size="10 MB"/>
			</Policies>
			<DefaultRolloverStrategy max="5"/>
		</RollingFile>
	</Appenders>
	<Loggers>
		<!-- Validation logging -->
		<Logger name="VALIDATION" level="INFO" additivity="false">
			<AppenderRef ref="lqeValidationTaskLog"/>
		</Logger>
	</Loggers>
</Configuration>

About this task

Engineering Lifecycle Management application administrators might ask you to validate the LQE or LDX index after they validate TRS feeds.

Consider the following points before you validate the LQE index:
  • Run this task when system load is low. The time that is required for validation depends on the size of the index.
  • Don’t validate all the TRS data providers at the same time. Validate large data providers on separate days.
  • Validate only one TRS feed for one data provider (Engineering Lifecycle Management application) at a time. For Engineering Lifecycle Management applications that have more than one TRS feed, run a separate validation for each feed.
  • Administrators can run LQE or LDX validation regularly to confirm whether the LQE or LDX data is in sync with the TRS feed from the data provider. For example, if you run weekly validation and resources of the same type are logged as missing, you must check the data-providing application. You might need to change or configure a setting in that application. If the validation task runs for a few months without reporting any issues, you can extend the validation schedule. For example, you can run the task monthly.

To stop a validation that is running, restart the LQE or LDX server.

Procedure

  1. Open the Data Providers page: https://host:port/lqe/web/admin/data-sources
  2. Click the name of the TRS feed to validate.
  3. Click Validate artifacts and configurations in the header of the page.
  4. Optional: Choose when to validate the data provider by selecting a date and time.
  5. Click Validate artifacts and configurations and follow the prompts.

    While LQE or LDX reads the TRS feed, you see this message: Waiting for a response from the data provider. This step can take a while, depending on the number of resources in the feed. This is an expected behavior, and no action is needed.

  6. Optional: Schedule LQE or LDX validation activities regularly.
    1. Click Edit Schedule in the Validation Schedule section.
    2. Select the Enable Schedule checkbox.
    3. Specify the time and days of the week when you want to run the validation.
    4. Save your changes.

    When you enable a validation schedule for a TRS feed and it is the next scheduled task, the scheduled validation time is displayed in the Scheduled Tasks column of the Data Providers page.

Results

When the validation finishes, the results are published on the Data Providers page, at https://host:port/lqe/web/admin/data-sources. Click the status link of a data provider:

On the Data Provider Status page, you can see how many resources were updated (because they were either missing or stale), and how many extra resources were removed.

On the Overview tab, click Download Logs to retrieve all the LQE or LDX logs for this data provider.

The validation log files are named lqe-validation-<encoded_data_source_URL>.log. The logs list resource discrepancies in the following categories:
  • Missing resources: These resources are included in the TRS feed but not in the LQE or LDX index. The validation process fetches these resources again to resolve the discrepancy.
  • Stale resources: The resources in the TRS feed are newer than the resources in the LQE or LDX index. The validation process fetches these resources again to resolve the discrepancy.
  • Extra resources: These resources are included in the LQE or LDX index, but not in the TRS feed. The validation process deletes these resources to resolve the discrepancy.

You can enable email notification for validation events. For more information, see Setting up email notification for Lifecycle Query Engine events.

If you still see discrepancies in the content of a report after validation, consider reindexing.

After you click Validate artifacts and configurations, no further changes are indexed for that data provider. If you scheduled validation to run later, no changes are processed from the moment you click the validation button, until the validation finishes.

For more information, watch the following video: