Manta Flow Collibra Integration Constraints

There are several constraints applied by default that the user must not violate in order for the IBM Automatic Data Lineage integration with Collibra to work as intended.

Constraints Caused by the Import Synchronization Process

Constraint Description

Communities used by Automatic Data Lineage are not be renamed If any community in DGC was used for any export from Automatic Data Lineage to Collibra and the community is renamed in DGC, the next export will effectively truncate the content of the old community and recreate all assets in a new community with the original name.
This also affects the performance of the subsequent export—it can take several times longer than it should.
This behavior can be avoided by following these steps.
- The Single Synchronization ID property (Manta Flow Collibra Client Configuration) has to be set to false.
- The Export External Mappings property (Manta Flow Collibra Client Configuration) has to be set to false.
- The information about already-finished exports for the original community has to be removed. This can be done before or after the rename operation is performed but before the next export to the original community is executed. To remove the information:
1. Get the synchronizationId of the previous export. This information can be obtained from Automatic Data Lineage log files or from the Collibra Import API endpoint GET /import/synchronize (the request can be sent via https://<collibra_url>/docs/rest-importer/index.html#/Import/findSynchronizationInfos). A list of existing synchronization IDs is returned in the answer. The synchronization ID that is needed contains the name of the community in which Automatic Data Lineage uploaded the metadata.
2. Remove the export information by calling the Collibra Import API endpoint DELETE /import/synchronize/{synchronizationId} (where synchronizationId is the ID acquired in the previous step). The request can be sent via https://<collibra_url>;/docs/rest-importer/index.html#/Import/removeSynchronization.
If these steps are followed, the original community can be renamed and a new community with the original name can be created. The next export to the new community (having the original name) will be finished without having any effect on the original community (that is now renamed).

Domains used/created by Automatic Data Lineage should not be renamed/moved The behavior is the same as for Communities, just in a smaller scope.
This behavior cannot be avoided by any configuration change at the moment.

Constraint	Description
Communities used by Automatic Data Lineage are not be renamed	If any community in DGC was used for any export from Automatic Data Lineage to Collibra and the community is renamed in DGC, the next export will effectively truncate the content of the old community and recreate all assets in a new community with the original name. This also affects the performance of the subsequent export—it can take several times longer than it should. This behavior can be avoided by following these steps. - The Single Synchronization ID property (Manta Flow Collibra Client Configuration) has to be set to false. - The Export External Mappings property (Manta Flow Collibra Client Configuration) has to be set to false. - The information about already-finished exports for the original community has to be removed. This can be done before or after the rename operation is performed but before the next export to the original community is executed. To remove the information: 1. Get the `synchronizationId` of the previous export. This information can be obtained from Automatic Data Lineage log files or from the Collibra Import API endpoint GET `/import/synchronize` (the request can be sent via `https://<collibra_url>/docs/rest-importer/index.html#/Import/findSynchronizationInfos`). A list of existing synchronization IDs is returned in the answer. The synchronization ID that is needed contains the name of the community in which Automatic Data Lineage uploaded the metadata. 2. Remove the export information by calling the Collibra Import API endpoint DELETE `/import/synchronize/{synchronizationId}` (where `synchronizationId` is the ID acquired in the previous step). The request can be sent via `https://<collibra_url>;/docs/rest-importer/index.html#/Import/removeSynchronization`. If these steps are followed, the original community can be renamed and a new community with the original name can be created. The next export to the new community (having the original name) will be finished without having any effect on the original community (that is now renamed).
Domains used/created by Automatic Data Lineage should not be renamed/moved	The behavior is the same as for Communities, just in a smaller scope. This behavior cannot be avoided by any configuration change at the moment.

Constraints Caused by External Mappings

You can change the Automatic Data Lineage configuration to get rid of these constraints. This can be set in Manta Admin UI. Set Export External Mappings to False on the configuration page Configurations > Integrations > Collibra > Collibra Export.

When you're using this configuration, if the steps described for each of the constraints are only performed in Collibra DGC and are not reflected in the Automatic Data Lineage configuration, all of those changes will be overridden during the next export from Automatic Data Lineage.

Constraint	Description	Suggested solution
Communities used by Automatic Data Lineage cannot be renamed	If any community in DGC was used for any export from Automatic Data Lineage to Collibra, the community cannot be renamed in DGC. If such a community is renamed, the next export that tries to use either the old or the new community name will fail.	The user can create a new community with a new name and configure Automatic Data Lineage to use this new community instead of the old one. If Automatic Data Lineage is configured to use a single synchronization ID for all communities, the old community will be truncated* once the export process configured as described above is executed and finished.
Domains used/created by Automatic Data Lineage cannot be renamed	If any domain in DGC was used/created by any export from Automatic Data Lineage to Collibra, the domain cannot be renamed in DGC. If such a domain is renamed, the next export that tries to use either the old or the new domain name will fail.	The user can configure Automatic Data Lineage to use the new domain name instead of the old one. The old domain will be deleted once the export process configured as described above is executed and finished.
Domains used/created by Automatic Data Lineage cannot be moved between communities	If any domain in DGC was used/created by any export from Automatic Data Lineage to Collibra, the domain cannot be moved to another community in DGC. If such a domain is moved, the next export that tries to use either the old or the new community and domain name combination will fail.	The user can create a new community with a new name and configure Automatic Data Lineage to use this new community and the old domain names. If Automatic Data Lineage is configured to use a single synchronization ID for all communities, the domains in the old community will be deleted once the export process configured as described above is executed and finished.
Assets created by Automatic Data Lineage cannot be renamed	If any asset in DGC was created by any export from Automatic Data Lineage to Collibra, the asset cannot be renamed. If such an asset is renamed, the next export that tries to use either the old or the new asset name will fail.	The naming convention of assets is defined by Automatic Data Lineage. Users can partly customize this naming convention using the collibraMantaMapping.csv and collibraFilesystemMapping.csv configuration files in Automatic Data Lineage. These configuration files allow the user to manipulate the names of servers, databases, buckets, folders, and files.
Assets created by Automatic Data Lineage cannot be moved between communities	If any asset in DGC was created by any export from Automatic Data Lineage to Collibra, the asset cannot be moved to another community in DGC.	The user can create a new community with a new name and configure Automatic Data Lineage to use this new community to export particular assets. If Automatic Data Lineage is configured to use a single synchronization ID for all communities, the assets in the old community will be deleted once the export process configured as described above is executed and finished.

* A community is truncated when all domains, assets, and complex relations created in the community by Automatic Data Lineage are deleted.