Configuring Data Virtualization connections for lineage imports (MANTA Automated Data Lineage)

To import lineage for assets from Data Virtualization connections, complete several configuration steps.

Prerequsites

To import lineage, ensure that:

  • You have Data Virtualization instance on Cloud Pak for Data 5.2.0 or later with Federal Information Processing Standards (FIPS) disabled.

  • Data Virtualization Admin grants you DV_METADATA_READER role by using GRANT ROLE DV_METADATA_READER TO USER/ROLE/GROUP <auth_id> SQL statement. The following <auth_id> values are valid for ROLE:

    • DV_ADMIN
    • DV_STEWARD
    • DV_ENGINEER
    • DV_USER

  • You have the Admin or Steward role in Data Virtualization or the SELECT privilege on all Data Virtualization objects or schemas and their dependencies participating in the lineage import.

    Alternatively, you can disable the Restrict visibility option by using Managing the visibility of virtual objects in Data Virtualization in the Cloud Pak for Data documentation.

  • You use either the username and password or API key as connection credentials to extract lineage from Data Virtualization. Use my platform credentials option does not work for lineage extraction.

  • You have the INSPECT privilege on all data sources in Data Virtualization that your scanned virtual tables originate from. Alternatively, the INSPECT privilege can be granted on each data source in Data Virtualization to the DV_METADATA_READER role.

Restrictions

  • You only can extract lineage of virtualized tables from relational data sources. Object store and virtualized files are not available.
  • Virtual-to-source lineage is not connected when the source object is an alias.
  • Virtual-to-source lineage is not connected when multiple objects with the same name, for example a table and a function, exist within the same schema at the data source.
  • Lineage is not a shared asset property, so catalog and project asset duplicates will not share it.
  • Assets imported by using lineage metadata import will not be de-duplicated with the assets published from Data Virtualization. New data assets will be created in the target catalog.
  • Assets with lineage will only be imported if another connected asset is also included in the same job. For example, a virtual table won’t be imported on its own unless its data source or a derived view is included in the import as well.

Procedure

  1. In the project where metadata lineage import job will be executed, create a new independent Data Virtualization connection.

  2. Download Data Virtualization SSL certificate from CP4D Menu > Data > Data virtualization > Menu > Configure connection > Download SSL Certificate.

  3. In the project where you run MDI Lineage job, create a new non-Platform Data Virtualization connection with exactly the same details as the Data Virtualization Platform connection in CP4D Menu > Data > Connections and enter the SSL certificate contents in the SSL certificate field.

  4. Use that connection to run metadata lineage import job.