Manta Flow Alation: IBM Automatic Data Lineage-Alation Resource Mapping

Alation Resource Mapping Configuration

It is possible, and usually necessary, to map database objects in the Manta Flow metadata repository (tables, views, etc.) to Alation Data Catalog resources. This mapping can be set on the Manta Admin UI screen under Configurations > Integrations > Alation > <technology> Alation Mapping, where <technology> represents the technology the mapping is made for. Although the mapping is different for each technology, there are some common principles.

1. The first column always represents the Alation source ID or BI server ID. It is an output of the mapping.

2. The other columns each define a set of database or BI objects belonging to the Alation resource. They are an input of the mapping. The object set definition for each technology is described below.

3. The entries are case sensitive — for example, HostAndInstanceName, Host, Global Database Name, etc. must be specified with the exact case as internally defined in the database itself; otherwise, the mapping will not be applied.

Db2 Resource Mapping Configuration

The Db2 Alation mapping structure:

*For a description of how to get the Alation source ID, see How to Acquire an Alation BI Server ID.

The example mapping (see the values in the table above) says that the resource with the ID 1 should cover the database objects of the host db2server, instance inst, database PROD, and schemas starting with PROD_MART_ except for PROD_MART_TEST.

MS SQL Resource Mapping Configuration

The MS SQL Alation mapping structure:

The example mapping (see the values in the table above) says that the resource with the ID 2 should cover the database objects of the server 192.168.25.1, instance SQL08, database mart, and all schemas except the ones starting with test_.

Netezza Resource Mapping Configuration

The Netezza Alation mapping structure:

The example mapping (see the values in the table above) says that the resource with the ID 3 should cover the database objects of the server nzserver, database MART, and schema ADMIN.

Oracle Resource Mapping Configuration

The Oracle Alation mapping structure:

The example mapping (see the values in the table above) says that the resource with ID 4 should cover the database objects of the global database orcl.manta.eu and schemas starting with PROD_MART_ except PROD_MART_TEST.

Teradata Resource Mapping Configuration

The Teradata Alation mapping structure:

The example mapping (see the values in the table above) says that the resource with the ID 5 should cover the database objects of the server 127.0.0.1 and databases starting with PROD_MART_ except PROD_MART_TEST.

PostgreSQL Resource Mapping Configuration

The PostgreSQL Alation mapping structure:

The example mapping (see the values in the table above) says that the resource with the ID 6 should cover the database objects of the host pghost, database martdb, and schema public.

SSAS Resource Mapping Configuration

The SSAS Alation mapping structure:

*For a description of how to get the Alation BI server ID, see How to Acquire an Alation BI Server ID.

The example mapping (see the values in the table above) says that the resource with the ID 11 should cover the database objects of the server SQLServer and the database or tabular model ModelDB.

Snowflake Resource Mapping Configuration

The Snowflake Alation mapping structure:

The example mapping (see the values in the table above) says that the resource with the ID 8 should cover the database objects of the account mantapartner, region us-east-1, database PROD_MART, and schemas DWH and PUBLIC.

BigQuery Resource Mapping Configuration

The BigQuery Alation mapping structure:

The example mapping (see the values in the table above) says that the resource with the ID 17 should cover the database objects of the service URL https://www.googleapis.com/bigquery/v2, project PROD_MART, and datasets DWH and PUBLIC.

SAP HANA Resource Mapping Configuration

The SAP HANA Alation mapping structure:

The example mapping (see the values in the table above) says that the resource with the ID 17 should cover the database objects of the host hostname1 and database dev1, including all schemas except schema1 and schema2.

BI Technologies (Reporting Tools) Resource Mapping Configuration

There is a common configuration structure for all supported reporting tools. The <technology> Alation mapping structure:

The example mapping (see the values in the table above) says that the resource with the ID 12 should cover the BI objects of the obiee connection.

Custom Database Resource Mapping Configuration (Available as of R42.5)

There is a common configuration structure for all supported reporting tools. The <technology> Alation mapping structure:

The example 1 mapping (see the values in the preceding table) says that the resource with the ID 12 should cover the ORCL custom database objects of the 127.0.0.1 server node, excluding schema1.

The example 2 mapping (see the values in the preceding table) says that the resource with the ID 12 should cover the ORCL custom database objects. This is valid in cases were the Server node is not present.

Filesystem resource mapping configuration (available as of R42.12)

The Filesystem Alation mapping structure is as follows:

The example 1 mapping (see the values in the preceding table) shows that the Alation source with the ID 1 should cover file assets in S3 Manta resource, on /myBucket base path, in folders starting with prod. The file assets are also uploaded into the Alation source.

The example 2 mapping (see the values in the preceding table) shows that the Alation source with the ID 1 should cover file assets in S3 Manta resource, on /myBucket base path, in all folders except the ones starting with doc, temp, or test. The file assets aren't uploaded into Alation and are expected to already exist in the Alation source.

SAS resource mapping configuration

Available as of R42.13

The SAS Alation mapping structure is as follows:

Examples

A metadata repository might have the following structure:

sas [Connection]
|
+- myProgram [SAS Program]
   |
   +- pilot [SAS Library]
   |  |
   |  +- DELIVERY [Table]
   |  |
   |  +- ORDER [Table]
   |
   +- production [SAS Library]
   |  |
   |  +- DELIVERY [Table]
   |  |
   |  +- ORDER [Table]
   |
   +- temp [SAS Library]
   |  |
   |  +- TRASH [Table]
   |
   +- test [SAS Library]
      |
      +- DELIVERY [Table]
      |
      +- ORDER [Table]

When example 1 mapping applies (see the values in the preceding table), the production SAS Library under sas/myProgram SAS Program is included in the export. The DELIVERY and ORDER tables are also uploaded into the Alation source of ID 1.

When example 2 mapping applies (see the values in the preceding table), the temp and test SAS Library nodes are excluded from the export, so the pilot and production libraries under sas/myProgram SAS Program are exported. However, the SAS tables upload is disabled, so the DELIVERY and ORDER tables must be extracted by a native Alation data source of ID 1, and Automatic Data Lineage uploads only lineage to them.

Available as of R42.14

The SAS Alation mapping structure is as follows:

Examples

A metadata repository might have the following structure:

SAS
|
+- localhost [Server]
   |
   +- pilot [SAS Directory]
   |  |
   |  +- DELIVERY [Table]
   |  |
   |  +- ORDER [Table]
   |
   +- production [SAS Directory]
   |  |
   |  +- DELIVERY [Table]
   |  |
   |  +- ORDER [Table]
   |
   +- temp [SAS Directory]
   |  |
   |  +- TRASH [Table]
   |
   +- test [SAS Directory]
      |
      +- DELIVERY [Table]
      |
      +- ORDER [Table]

When example 1 mapping applies (see the values in the preceding table), the production SAS Directory under localhost Server is included in the export. The DELIVERY and ORDER tables are also uploaded into the Alation source of ID 1.

When example 2 mapping applies (see the values in the preceding table), the temp and test SAS Directory nodes are excluded from the export, so the pilot and production directories under localhost Server are exported. However, the SAS tables upload is disabled, so the DELIVERY and ORDER tables must be extracted by a native Alation data source of ID 1, and Automatic Data Lineage uploads only lineage to them.

How to Acquire an Alation Source ID

There are several places where the source ID value can be found.

The first option is to find the ID in the Alation UI. It is not explicitly rendered in the content of the screen, but the URL links leading to the source detail screen contain the ID. It is possible to go to the source overview screen and read the value from the URL or to hover over the link and read the value from the link URL that is typically shown in the lower-left corner.

Another option is to use Alation APIs to get the list of sources. The API endpoint [GET] http://<ALATION_URL>/integration/v1/datasource can be used for this. The output (see the example below) contains both the Title and the ID of the BI server.

[   
...
    {
        "id": 9,
        "title": "Oracle",
        ...
    },
...
]

How to Acquire an Alation BI Server ID

There are several places where the BI server ID value can be found.

The first option is to find the ID in the Alation UI. It is not explicitly rendered in the content of the screen, but the URL links leading to the BI server detail screen contain the ID. It is possible to go to the BI server overview screen and read the value from the URL or to hover over the link and read the value from the link URL that is typically shown in the lower-left corner.

Another option is to use Alation APIs to get the list of BI servers. The API endpoint [GET] http://<ALATION_URL>/integration/v2/bi/server can be used for this. The output (see the example below) contains both the Title and the ID of the BI server.

[   
...
    {
        "id": 25,
        "title": "SAP BO:default",
        ...
    },
...
]