Manta Flow Informatica EDC Resource Mapping
Manta Resource Name Configuration
By default, the Manta scripts resource name is
${< technology >.dictionary.id}Scripts and the Manta links resource name is ${< technology >.dictionary.id}Links. These values can be configured by setting the manta.resource.iedc.scripts and
manta.resource.iedc.links properties on the Manta Admin UI screen under Configurations > Integrations > IEDC > IEDC Export Common, where < technology > represents the technology the resource names are configured
for.
Informatica EDC Resource Mapping Configuration
It is possible and usually necessary to map database objects in the Manta Flow metadata repository (tables, views, etc.) to Informatica EDC resources. This mapping can be set on the Manta Admin UI screen under
Configurations > Integrations > IEDC > \< technology > IEDC Mapping, where < technology > represents the technology the mapping is made for. Although the mapping is different for each technology, there are some
common principles. ń
-
The first column always represents the Informatica EDC resource. It is an output of the mapping.
-
The other columns each define a set of database objects belonging to the Informatica EDC resource. They are an input of the mapping. The object set definition for each technology is described below.
-
The entries are case sensitive; for example,
HostAndInstanceName,Host,Global Database Name, etc. must be specified with the exact case as internally defined in the database itself, otherwise the mapping will not be applied. -
If no entry matches, then the default dictionary mapping will be used, so the dictionary ID will be used as the target EDC resource name. This means that everything that’s available in the Manta repository for the selected technology and connection will be exported into EDC.
Db2 Resource Mapping Configuration
The Db2 IEDC mapping structure:
| Column | Description | If empty | Example value |
|---|---|---|---|
| IEDC Resource | The mapped Informatica EDC resource | Cannot be empty | DB2Mart |
| HostAndInstanceName | Name of the host and instances of the resource database objects, separated by backslashes (\) |
All objects of the processed connection belong to the resource | db2server\\inst |
| DatabaseName | Name of the database of the resource database objects | All objects of the given host and instances belong to the resource | PROD |
| IncludeSchemas | Regular expression of the database schemas of the resource database objects | All objects of the given database belong to the resource except the ones defined in ExcludeSchemas |
PROD_MART_.* PROD_MRT |
| ExcludeSchemas | Regular expression of the database schemas with database objects excluded from the resource | No objects are explicitly excluded from the resource | PROD_MART_TEST |
The example mapping (see the values in the table above) says that the resource DB2Mart should cover the database objects of the host db2server, instance inst, database PROD, and schemas starting
with PROD_MART_ except PROD_MART_TEST.
MS SQL Resource Mapping Configuration
The MS SQL IEDC mapping file structure:
| Column | Description | If empty | Example value |
|---|---|---|---|
| IEDC resource | The mapped Informatica EDC resource | Cannot be empty | MSSQLMart |
| Host | Backslash-separated (\) name of the server and instances of the resource database objects |
All objects of the connection processed belong to the resource | 192.168.25.1\\SQL08 |
| Included database/schemas | Slash-separated (/) regular expressions of the databases and schemas of the resource database objects |
All objects of the given server and instances belong to the resource except the ones defined in Excluded Schemas | mart/.* mrt/.* |
| Excluded database/schemas | Slash-separated (/) regular expressions of the databases and schemas with database objects excluded from the resource |
No objects are explicitly excluded from the resource | mart/test_.* |
The example mapping (see the values in the table above) says that the resource MSSQLMart should cover the database objects of the server 192.168.25.1, instance SQL08, database mart, and all schemas
except the ones starting with test_.
Netezza Resource Mapping Configuration
The Netezza IEDC mapping structure:
| Column | Description | If empty | Example value |
|---|---|---|---|
| IEDC resource | The mapped Informatica EDC resource | Cannot be empty | NetezzaMart |
| Server name | Name of the server of the resource database objects | All objects of the connection processed belong to the resource | nzserver |
| Database name | Name of the database of the resource database objects | All objects of the server belong to the resource | MART |
| Include schemas | Regular expression of the schemas of the resource database objects | All objects of the given server and instances belong to the resource except the ones defined in ExcludeSchemas |
ADMIN |
| Exclude schemas | Regular expression of the schemas with database objects excluded from the resource | No objects are explicitly excluded from the resource |
The example mapping (see the values in the table above) says that the resource NetezzaMart should cover the database objects of the server nzserver, database MART, and schema ADMIN.
Oracle Resource Mapping Configuration
The Oracle IEDC mapping structure:
| Column | Description | If empty | Example value |
|---|---|---|---|
| IEDC resource | The mapped Informatica EDC resource | Cannot be empty | OracleMart |
| Global database name | Name of the database of the resource database objects | All objects of the connection processed belong to the resource | orcl.manta.eu |
| Included schemas | Regular expression of the schemas of the resource database objects | All objects of the given database belong to the resource except the ones defined in Excluded Schemas |
PROD_MART_.* PROD_MRT |
| Excluded schemas | Regular expression of the schemas with database objects excluded from the resource | No objects are explicitly excluded from the resource | PROD_MART_TEST |
The example mapping (see the values in the table above) says that the resource OracleMart should cover the database objects of the global database orcl.manta.eu and schemas starting with PROD_MART_ except
PROD_MART_TEST.
EDC provides an option for users to configure the identifier to be used for an Oracle database name by an EDC-native Oracle resource. By default, it is the instance name. However, the user can chose the database name by setting -DextractDatabaseName=true in the JVM optionsconfiguring. In the latter case, this change needs to be reflected in Manta as well as in the Oracle scanner common configuration in the property oracle.instanceNameQuery.
Teradata Resource Mapping Configuration
The Teradata IEDC mapping file structure:
| Column | Description | If empty | Example value |
|---|---|---|---|
| IEDC resource | The mapped Informatica EDC resource | Cannot be empty | TeradataMart |
| Teradata server name | Name of the server of the resource database objects | All objects of the connection processed belong to the resource | 127.0.0.1 |
| Included databases | Regular expression of the databases of the resource database objects | All objects of the given database belong to the resource except the ones defined in Excluded Databases |
PROD_MART_.* PROD_MRT |
| Excluded databases | Regular expression of the databases with objects excluded from the resource | No objects are explicitly excluded from the resource | PROD_MART_TEST |
The example mapping (see the values in the table above) says that the resource TeradataMart should cover the database objects of the server 127.0.0.1 and databases starting with PROD_MART_ except
PROD_MART_TEST.
PostgreSQL Resource Mapping Configuration
The PostgreSQL IEDC mapping structure:
| Column | Description | If empty | Example value |
|---|---|---|---|
| IEDC resource | The mapped Informatica EDC resource | Cannot be empty | PostgreMart |
| Host | Name of the host of the resource database objects | All objects of the connection processed belong to the resource | pghost |
| Database | Name of the database of the resource database objects | All objects of the host belong to the resource | martdb |
| Include schemas | Regular expression of the schemas of the resource database objects | All objects of the given host and database belong to the resource except the ones defined in Exclude Schemas |
public |
| Exclude schemas | Regular expression of the schemas with database objects excluded from the resource | No objects are explicitly excluded from the resource |
The example mapping (see the values in the table above) says that the resource PostgreMart should cover the database objects of the host pghost, database martdb, and schema public.
SSAS Resource Mapping Configuration
The SSAS IEDC mapping structure:
| Column | Description | If empty | Example value |
|---|---|---|---|
| IEDC resource | The mapped Informatica EDC resource | Cannot be empty | SSASTabularScripts |
| Server name | Name of the server of the resource objects | All objects of the connection processed belong to the resource | SQLServer |
| Included databases / tabular models | Regular expression of the databases or tabular models of the resource objects | All objects of the given server belong to the resource except the ones defined in Excluded Databases / Tabular Models |
ModelDB |
| Excluded databases / tabular models | Regular expression of the databases or tabular models excluded from the resource | No objects are explicitly excluded from the resource |
The example mapping (see the values in the table above) says that the resource SSASTabularScripts should cover the database objects of the server SQLServer and the database or tabular model ModelDB.
Snowflake Resource Mapping Configuration
The Snowflake IEDC mapping structure:
| Column | Description | If empty | Example value |
|---|---|---|---|
| IEDC resource | The mapped Informatica EDC resource | Cannot be empty | SnowflakeMart |
| Account Name | Name of the account of the resource database objects | All objects of the connection processed belong to the resource | mantapartner |
| Region | Name of the region of the resource database objects | All objects of the account with an unspecified region belong to the resource | us-east-1 |
| Included Databases and Schemas | Slash-separated (/) regular expressions of the databases and schemas of the resource database objects |
All objects of the given server and instances belong to the resource except the ones defined in Excluded Schemas | PROD_MART/DWH,PROD_MART/PUBLIC |
| Excluded Databases and Schemas | Slash-separated (/) regular expressions of the databases and schemas with database objects excluded from the resource |
No objects are explicitly excluded from the resource |
The example mapping (see the values in the table above) says that the resource SnowflakeMart should cover the database objects of the account mantapartner, region us-east-1, database PROD_MART,
and schemas DWH and PUBLIC.
BigQuery Resource Mapping Configuration
The BigQuery IEDC mapping structure:
| Column | Description | If empty | Example value |
|---|---|---|---|
| IEDC resource | The mapped Informatica EDC resource | Cannot be empty | BigQueryMart |
| Service URL | URL of the BigQuery service endpoint | All objects of the service URL belong to the resource | https://www.googleapis.com/bigquery/v2 |
| Included projects/datasets | Slash-separated (/) regular expressions of the projects and datasets of the resource database objects |
All objects of the service URL belong to the resource except the ones defined in Excluded projects/datasets | PROD_MART/DWH,PROD_MART/PUBLIC |
| Excluded projects/datasets | Slash-separated (/) regular expressions of projects and datasets with database objects excluded from the resource |
No objects are explicitly excluded from the resource |
The example mapping (see the values in the table above) says that the resource BigQueryMart should cover the database objects of the service URL https://www.googleapis.com/bigquery/v2, project PROD_MART, and
datasets DWH and PUBLIC.
SAP HANA Resource Mapping Configuration
The SAP HANA IEDC mapping structure:
| Column | Description | If empty | Example value |
|---|---|---|---|
| IEDC resource | The mapped Informatica EDC resource | Cannot be empty | SapHanaResource |
| Host | Hostname of the server | All objects of the hosts belong to the resource | hostname1 |
| Database | Name of the database instance | All objects of the databases belong to the resource | dev1 |
| Included schemas | List of schemas to extract, separated by commas; each item is evaluated as a regular expression | All objects of the host belong to the resource except the ones defined in Excluded Schemas | .* |
| Excluded schemas | List of schemas to be excluded, separated by commas. Each item is evaluated as a regular expression. If left blank, no schemas will be excluded. | No objects are explicitly excluded from the resource | schema1,schema2 |
The example mapping (see the values in the table above) says that the resource SapHanaResource should cover the database objects of the host hostname1 and database dev1, including all schemas except
schema1 and schema2.
Filesystem Resource Mapping Configuration
The Filesystem IEDC mapping structure:
|
Column |
Description |
If empty |
|---|---|---|
|
IEDC resource |
The mapped Informatica EDC resource Optionally, it can also specify a path to the mapped EDC directory in the EDC File System resource. In this case, the path to the included Manta directory will be replaced by the EDC directory path within the mapping. If the directory is not specified, the Manta resource name will simply be replaced by the EDC resource name. |
Cannot be empty |
|
FileSystem resource name |
Filesystem resource name used in Manta Flow; by default |
Cannot be empty |
|
Included directories |
List of directory paths, separated by commas, which should be in this resource; each path entry is evaluated as a regular expression |
All directories and files in the given path belong to the resource except the ones defined in
|
|
Excluded directories |
List of directory paths, separated by commas, which should not be in this resource; each path entry is evaluated as a regular expression |
No objects are explicitly excluded from the resource |
Example:
| IEDC resource | FileSystem resource name | Included directories | Excluded directories |
|---|---|---|---|
FileServerSimple |
Filesystem |
c:/documents/poc |
|
FileServerDev |
Filesystem |
c:/documents/dev/.* |
|
FileServerProd://docs/users |
Filesystem |
c:/documents/users |
c:/documents/users/internal |
FileServerTest://docs/users/internal |
Filesystem |
c:/documents/users/internal/test |
Mapping examples:
| Manta file (mapping source) | EDC file (mapping target) | Notes |
|---|---|---|
| /Filesystem/c:/documents/poc/statistics.txt | FileServerSimple://c:/documents/poc/statistics.txt | Filesystem is just replaced by FileServerSimple because the FileServerSimple resource has not attached a directory path. |
| /Filesystem/c:/documents/dev/tutorials/java/classes.txt | FileServerDev://java/classes.txt | /Filesystem/c:/documents/dev/tutorials is replaced by FileServerDev because the classes.txt Manta file is in the Filesystem resource and the c:/documents/dev/tutorials directory
matching the included directory pattern c:/documents/dev/.*;. |
| /Filesystem/c:/documents/users/external/list/2020.txt | FileServerProd://docs/users/list/2020.txt | /Filesystem/c:/documents/users is replaced by FileServerProd://docs/usersbecause the 2020.txt Manta file is in the Filesystem resource and the included directoryc:/documents/users and is not in the excluded directory c:/documents/users/internal. |
| /Filesystem/c:/documents/users/internal/test/list/2019.txt | FileServerTest://docs/users/internal/list/2019.txt | /Filesystem/c:/documents/users/internal/test is replaced by FileServerTest://docs/users/internal because 2019.txt the Manta file is in the Filesystem resource and the included directoryc:/documents/users/internal/test. |
| /Filesystem/c:/documents/users/internal/prod/list/2021.txt | /Filesystem/c:/documents/users/internal/prod/list/2021.txt | Fallback — the 2021.txt Manta file is not mapped because it is in the included directory c:/documents/users as well as in the excluded one c:/documents/users/internal. |