Getting lineage for business intelligence reports (Watson Knowledge Catalog)

Capture end-to-end data lineage for business intelligence reports. Add business data lineage to business intelligence assets and, optionally, to imported data assets in your catalogs, and access detailed technical data lineage in MANTA Automated Data Lineage.

Lineage for business intelligence reports depicts the data flows that transform and populate the source data for the report. Import this information to visualize these data flows.

This import option is not available in projects that are marked as sensitive.

Before you import metadata, design your metadata import to ensure that you understand all your options and make the appropriate choices for your goals. For more information, see Designing metadata imports.

To retrieve the list of supported connections or to create a metadata import asset, you can use APIs instead of the user interface. The links to these APIs are listed in the Learn more section.

To run metadata enrichment on data assets that were added with a lineage import for business intelligence reports, make the data assets available in a project. For more information, see Adding catalog assets to a project.

Asset types
Business intelligence assets that represent components of business intelligence reports. Optionally, data assets that serve as the source or target of business intelligence reports. For more information, see Asset types created through metadata import.
Supported connections
For more information, see the Metadata import (discovery) and Metadata import (lineage) columns and the Other data sources section in Supported connectors.
Required permissions
To create, manage, and run a metadata import, you must have the following roles and permissions:
  • The Manage asset discovery user permission.
  • The Admin or the Editor role in the project.
  • The Admin or the Editor role in the catalog to which you want to import the assets.
  • Access to the connections to the data sources of the data assets to be imported and the SELECT or a similar permission on the corresponding databases.

Prerequisites

Before you can generate and import lineage for business intelligence reports, create an input file and upload it to your project. For more information, see Preparing manual input for importing business intelligence reports.

Creating a metadata import asset and generating or importing lineage metadata for business intelligence reports

To create a metadata import asset and a job for generating and importing technical and lineage metadata for business intelligence reports:

  1. Open a project, go to the project's Asset page, and click New asset > Metadata Import.

  2. Select the Get BI report lineage option. If you don't see this option, the Advanced metadata import feature is not enabled and no license key is installed. For more information, see Installed features and license requirements.

  3. Specify a name for the metadata import. Optionally, you can provide a description.

  4. Optional: To simplify searching, select tags to be assigned to the metadata import asset. To create new tags, enter the tag name and press Enter.

  5. Select a catalog from the list. If your project is marked as sensitive, you can't create and run data model imports.

    For this type of import, the import target can be only a catalog. For more information, see Scope of import.

  6. Define a scope for the metadata import. For more information, see Scope of import.

    1. Click Select scope. The following selections are valid.

      • You can select a single input file from your project to import reports from Microsoft SQL Server Analysis Services or Statistical Analysis System. To select the file from your project for the import, expand the Data asset node. For more information, see Preparing manual input for importing business intelligence reports.
      • You can select one or more connections to a reporting tool. To select a connection to a reporting tool, expand the Connection node.
      • In addition to an input file or connections to a reporting tool, you can select connections to data sources that hold source assets of the business intelligence report to import technical metadata for such assets and get the lineage between the report and these assets.
      • Also, you can combine a single input file with one ore more connections to a reporting tool and any number of connections to data sources.

      You can select connections that exist in the project. You can also click Create a new connection and create a connection asset. You can import metadata and lineage from the data sources that are listed in Supported connectors.

    2. Review the selected scope.

  7. Define whether you want to run scheduled import jobs. If you don't set a schedule, you run the import when you save the metadata import asset. You can rerun the import manually at any time. For more information, see Scheduling options.

  8. Optional: Customize the import behavior. You can choose to prevent specific properties from being updated and to delete existing assets that are not included in the reimport. For more information, see Advanced import options.

  9. Review the metadata import configuration. To make changes, click the edit (edit icon) icon on the tile and update the settings.

  10. Click Create. The metadata import asset is added to the project, and a metadata import job is created. If you didn't configure a schedule, the import is run immediately. If you configured a schedule, the import runs on the defined schedule.

    Important: If the business intelligence report was already imported through a different metadata import, it is not imported anew but is updated. The business intelligence assets no longer show up in the initial metadata import. Only the most recently run metadata import contains the assets.

    The same is true for data assets if you selected to add those to your lineage import.

Depending on the outcome of the metadata import job run, a completion message or an error notification is displayed.

A completion message is displayed when the job run completed successfully, completed with warnings, or completed with errors. An error notification is displayed if the entire job run failed. Either type of notification contains a link to the job run log that provides details about the specific job run.

Lineage imports are long-running processes. Don't expect immediate results. When the import is complete, the imported assets and their business data lineage are available in the catalog that you selected as the target one. The imported assets have an automatically assigned tag that reflects the originating reporting tool. The imported lineage is available on the asset's Lineage tab. More lineage information is available in MANTA Automated Data Lineage. You can access that information through the Go to asset's technical data lineage link in the About the asset panel.

Learn more

Next steps

Parent topic: Importing metadata