Azure DevOps Integration: Data Validation Report
Two-way data synchronization implies that both systems can be inaccessible for different periods of time and data can get out of sync. With the data validation report, you are able to find the items that are not synced properly and fix the consequences right from the report.
Who can access the report?
Targetproccess Administrators, as they have access to Targetprocess Settings –> Integrations and therefore to Data validation report tab in every issue level integration profile.
What errors could be found and fixed?
With the report, you can find three groups of errors or improper sync consequences.
- Conflicting fields: out-of-sync fields in all pairs of linked issues.
- Lost and not imported items: issues that match the integration settings and can be imported.
- Invalid pairs: pairs, where one of the issues does not exist or its ID/Type changed and issues were lost during sync.
Scopes of the report
The first option
Identify conflicts among the synchronized items fields will scan all shared
issues and detect mismatches in fields values between Targetprocess and Azure DevOps:
If you find that a scan is taking too long, you can terminate a running Data Validation Report scan and narrow the scope.
To bigger integrations additional filters are recommended. You can use DSL filters as in the Targetprocess boards and lists and scan conflicts in selected entities only.
For example, it could be ?ModifyDate == '20-Jul-2023'to find if there
are mismatches in the data updated on July 20, 2023 in Targetprocess.
Or it could be a WIQL fitler to select items for scanning from Azure DevOps perspective. Remember to use only content of the WHERE clause only:
[System.AreaPath] UNDER 'ADO_Project\ADO_Team'
In case a conflicting pair is valid,
fields values will be compared and non-matching fields will show the results in ‘Conflicting items’
tab: 
If the pair is not valid – one of the issues does not exist
or change its ID/Type and sync is not possible – it will appear in the ‘Invalid links’ tab of the
report: 
The next two report options are for
detecting not imported issues, which are two separate requests to Azure DevOps and
Targetprocess.
By default, with no filters added, the report will search across
all the projects selected in ‘Projects’ tab of the profile (static scopes). If JavaScript routing is
added, then all accessible projects will be scanned despite the projects referenced in JavaScript
mapping.
Use additional WIQL or DSL filters if your integration profile uses dynamic routings with an undefined number of Azure DevOps/Targetprocess projects, or if you wish to narrow down the scope of scanning.
Conflicts and resolutions
In this release, the report scans all existing pairs of linked issues and compares field values according to field mappings. ‘Conflicting items’ report tab shows all fields where values do not match when they should.
Fields mapped with JavaScript code extensions must have a custom comparator added otherwise they will be skipped from the report by default.
Warning for missing JS mappings comparators
The data validation report compares the values of the fields in both tools and detects mismatches.
- Blue: Full JS mappings
- Yellow: Incomplete mappings
This report’s tab shows all pairs where one or both issues do not exist – either deleted or their ID or type was changed and cannot proceed to sync.
This category of errors has the following actions:
- 'Refresh conflicts'
- 'Unlink'
All issues that match the integration profile filters at the moment of the scanning Targetprocess or Azure DevOps
To fix conflicts from the 'Not Imported' apply one of two actions – "Pull to Targetprocess" or "Push to Azure DevOps" to import all selected (filtered) entities to selected tool.
Filters and actions
All issues filtered out and viewable in the list are considered as a selected scope for any action. Should you need to apply an action to a specific row of the report, filter the specific row out and then apply changes to the only row left in the report.
In case conflicted fields have their values changed, then ‘Refresh conflicts’ action will bring the actual values. If conflict is not actual anymore, it will disappear from the report (mappings removed, issues unlinked) or change its state to ‘Resolved’ (issue imported)
This action moves conflict to the ‘Ignored’ state. In this state, conflict will not be processed – not refreshed, not fixed. Before applying any conflict resolution strategy, you need to move conflict back from ‘Ignored’ state to an actual.
‘Unignore conflicts’ triggers conflict refresh in the background. If conflicts were fixed indirectly in one of the systems, then field conflict will move to a valid state – ‘Resolved’, if values match at this time, or back to ‘Conflict’, if mismatch is valid.
Conflict resolution strategies
To fix the conflicts in the mapped fields, you can choose one of three strategies:
Out of two conflicting values, the latest one will be selected and applied to the matching filed in the other system.
???? Note: This strategy works for conflicts where both fields, Targetprocess and Azure DevOps, have last modified dates. If at least one field in the pair does not have history and last updated date, then only ‘Pull to Targetprocess‘ or ‘Push to Azure DevOps’ can be applied.
With this strategy, all values from Targetprocess fields will be sent to Azure DevOps.
In the ‘Not imported’ section, this will import all selected Targetprocess items to Azure DevOps.
This strategy considers Azure DevOps as a source of truth and rewrites Targetprocess fields with values from Azure DevOps.
In the ‘Not imported’ section, this will import all selected Azure DevOps issues items to Targetprocess.
Unlink is required if the entity has been synced and the link is not valid for various reasons (issue deleted or converted and its ID/Type changed and lost). In such cases, before importing an issue again as new it must be unlinked from the previous item.
As soon as the issues are unlinked, they move to the ‘Resolved’ state and no other actions are possible with resolved invalid links for now. The free issue can be imported manually to the connected tool.
Conflict resolution workflow
Before any action is applied, every field’s conflict is verified and its status and fields are updated. This validation can be called manually with the ‘Refresh’ action.
Status ‘Conflict’
Fields or issues that do not match.
With items in the ‘Conflict’ state, you can:
- Verify the conflict by applying ‘Refresh’ action
- ‘Ignore’ field and no action will apply to them, even if such issue gets under the filter
- Fix filtered conflicts with one of the resolving strategies – ‘Pull to Targetprocess’, ‘Push to Azure DevOps’, or ‘Latest values’
Temporary state which indicates that the existence of conflict is being checked. If values are the same, and if an action selected (’Unignore’, ‘Pull to Targetprocess’, ‘Push to Azure DevOps’, or ‘Latest values’) can be applied to selected items.
‘Processing’ state will automatically change
- To ‘Resolved’, if conflict is fixed
- Back to ‘Conflict’ if selected change cannot be applied or conflict still occurs and no further actions were applied
- To ‘Update Sent’ if action is applied and new value is sent to Azure DevOps
Final state. No actions will apply to resolved conflicts in current report version.
No actions will apply to conflicts in status ‘Ignored’. The ‘Unignore conflicts’ action will automatically call a ‘Refresh’ and actualize the conflict.
This state indicates that updates are in the queue and will be applied in Azure DevOps shortly. You must manually refresh conflicts in order to know if the changes applied successfully. This state will not change automatically to the next, even if values were updated already.
Cancel data validation report and conflicts resolution
- If the resolution process is no longer needed, canceling it can help prevent unnecessary processing and reduce the load on the system.
- If the resolution process is taking an excessively long time to complete, canceling it can help free up system resources and avoid delays.
