Teradata Resource Configuration
Before you configure your scanner, make sure you meet the prerequisites. Read our guide on Teradata integration requirements to double-check.
Source System Properties
This configuration can be setup by creating a new connection on Admin UI / Connections tab or editing an existing connection in Admin UI / Connections / Databases / Teradata / specific connection. New connection can also be created via Manta Orchestration API.
Granularity of the IBM Automatic Data Lineage connection for Teradata is one Teradata instance. Use list of databases to limit the scope of analysis as needed. Use of multiple connections against a single Teradata server may lead to within-system lineage not be connected properly.
|
Property name |
Description |
Example |
|---|---|---|
|
teradata.dictionary.id |
Name of the resource representing this Teradata database known as the dictionary ID, used as an output subdirectory name for extracted DDL files and the database dictionary |
Teradata |
|
teradata.extractor.host |
Name or IP address of the Teradata server |
teradata.lan, 127.0.0.1 |
|
teradata.extractor.port |
Port on which the Teradata server listens (non-SSL); see |
1025 |
|
teradata.extractor.username |
User name for connection to Teradata |
manta_user |
|
teradata.extractor.password |
User password for connection to Teradata |
manta_password |
|
teradata.extractor.dbsInclude |
The regular expression for database names which should be extracted; matching is case insensitive; leave it blank to extract each database from the defined connection |
|
|
teradata.extractor.dbsExclude |
The regular expression for database names which should NOT be extracted; matching is case insensitive |
|
|
teradata.extractor.ddl.dbsExclude |
The regular expression for database names from which DDL should NOT be extracted; this filter is used on the results of the filtering defined by the two previous regular expressions; matching is case insensitive This filter is useful when some databases are used as APIs or you would like to see them as a blackbox |
|
|
teradata.extraction.method |
Set to Agent:default when the desired extraction method is the default Manta Extractor Agent, set to Agent:{remote_agent_name} when a remote Agent is the desired extraction method, set to Git:{git.dictionary.id} when the Git ingest method is the desired extraction method. For more information on setting up a remote extractor Agent please refer to the Manta Flow Agent Configuration for Extraction documentation. For additional details on configuring a Git ingest method, please refer to the Manta Flow Agent Configuration for Extraction:Git Source documentation. |
default Git agent |
|
teradata.bteq.encoding |
BTEQ script encoding. See Encodings for applicable values. |
UTF-8 |
|
teradata.ddl.encoding |
DDL script encoding. See Encodings for applicable values. |
UTF-8 |
|
teradata.tpt.encoding |
TPT script encoding. See Encodings for applicable values. |
UTF-8 |
Common Scanner Properties
This configuration is common for all Teradata source systems and for all Teradata scenarios, and is configure in Admin UI / Configuration / CLI / Teradata / Teradata Common. It can be overridden on individual connection level.
|
Property name |
Description |
Example |
|---|---|---|
|
teradata.bteq.dir |
Directory of BTEQ scripts that work with the related Teradata database |
${manta.dir.input}/teradata/${teradata.dictionary.id}/bteq |
|
teradata.ddl.dir |
Directory of Teradata DDL scripts that work with the related database |
${manta.dir.temp}/teradata/${teradata.dictionary.id}/ddl |
|
teradata.tpt.dir |
Directory of TPT scripts that work with the related database |
${manta.dir.input}/teradata/ |
|
teradata.bteq.replace |
Path to the CSV file with the replacements to be applied to the provided BTEQ scripts; see Placeholder Replacement in Input Scripts for details about the replacement file format |
${manta.dir.input}/teradata/ |
|
teradata.bteq.replace.regex |
Flag specifying whether replacements for BTEQ scripts in the provided CSV file specified in
|
false |
|
teradata.tpt.replace |
Path to the CSV file with the replacements to be applied to the provided TPT scripts; see Placeholder Replacement in Input Scripts for details about the replacement file format |
${manta.dir.input}/teradata/ |
|
teradata.tpt.replace.regex |
Flag specifying whether replacements for TPT scripts in the provided CSV file specified in
|
false |
|
teradata.analyze.parallelCount |
Number of parallel threads which will analyze DDL, BTEQ, and TPT scripts |
4 |
|
filepath.lowercase |
Whether paths to files should be lowercase (false for case-sensitive file systems, true otherwise) |
false |
|
teradata.maxTime |
Timeout for parsing each Teradata script, in milliseconds |
600000 |
|
teradata.extractor.jdbcSuffix |
Enter possible additional JDBC configuration; the string has to start with a comma |
|
|
teradata.extractor.tmode |
Transaction mode of connection to the Teradata database |
ANSI, TERA, DEFAULT |
|
teradata.extractor.charset |
Session character set of connection to the Teradata database |
ASCII, LATIN1252_0A, UTF8 |
|
teradata.extractor.logmech |
Logon mechanism of connection to the Teradata database |
TD1, TD2, KRB5, LDAP |
|
teradata.extractor.dbPerQuerySet |
Number of databases that should be processed in one query set |
500 |
|
teradata.extractor.ssl.enable |
Enables SSL for the JDBC connection; if
Requires TeraJDBC driver version 17.10.00.07 or higher Due to limited support for custom truststore setup in TeraJDBC driver, a custom certificate / CA trusts and/or custom truststores need to be set at the level of the JVM initiating the JDBC connection (CLI/Agent). Automatic Data Lineage automatically uses port 443 for SSL-enabled connections (no additional configuration needed). Continue using
|
true false |
|
teradata.extractor.ssl.mode |
SSL mode (sets level of verification) Requires TeraJDBC driver version 17.10.00.07 or higher |
DISABLE |
|
teradata.extractor.encryptData |
Encrypts traffic between the DBMS and client Requires TeraJDBC driver version 15.00.00.15 or higher |
true false |
|
teradata.dictionary.mappingFile |
Path to automatically generated mappings for Teradata databases |
${manta.dir.temp}/teradata/teradataDictionaryMantaMapping.csv |
|
teradata.dictionary.mappingManualFile |
Path to mappings provided manually for Teradata databases |
${manta.dir.scenario}/conf/teradataDictionaryMantaMappingManual.csv |
|
teradata.expressionDescriptions.enabled.ddl |
Enables or disables building of transformation logic descriptions from SQL code in DDL scripts |
false |
|
teradata.expressionDescriptions.enabled.bteq |
Enables or disables building of transformation logic descriptions from SQL code in BTEQ scripts |
false |
|
teradata.expressionDescriptions.enabled.queryService |
Enables or disables building of transformation logic descriptions from Teradata SQL code defined in other technologies |
false |
|
teradata.tpt.variables.location |
Enter file containing manually provided values for TPT variables (see https://docs.teradata.com/r/Teradata-Parallel-Transporter-User-Guide/February-2022/Teradata-PT-Job-Components/Using-Job-Variables) The file content is in the format |
${manta.dir.input}/teradata/${dictionary.id}/variableValues.properties |
|
teradata.tpt.templates.location |
The path to sources of operator template overrides that is relative to
|
operators.zip |
|
teradata.tpt.includes.pathreplacements |
File with key=value replacements applied to the INCLUDE directive to adjust the path to included TPT script fragments so that the paths are relative and the included files are searched under
|
${manta.dir.input}/teradata/${teradata.dictionary.id}/replacements.properties |
|
teradata.analyze.retainUnusedResultSetColumns |
Flag specifying whether the data lineage should include sub-query resultset columns that do not have any downstream lineage By default, set to false |
false |
Note that all these properties can be moved to all source system specific property files.
Manual Mapping Properties
It is possible to manually configure mappings for Teradata databases in the file ${teradata.dictionary.mappingManualFile}listed in the table above. This is necessary for non-extracted schemas in the Automatic Data Lineage extraction
phase and to match names of database links in QueryGrid. In order to configure manual mapping, copy a relevant line from the automatically generated file ${teradata.dictionary.mappingFile} and modify the Connection ID column with the external connection identifier.
Each mapping has its own row with the following parts separated by semicolons.
| Property name | Description | Example |
|---|---|---|
| Dictionary ID | Name of the resource representing this Teradata server known as the dictionary ID, used as an output subdirectory name for extracted DDL files and the database dictionary | Teradata |
| Host name | Name of the host where the databases are running | 127.0.0.1 |
| Connection ID | External Teradata connection ID in third-party tools, for example, as it appears in the Informatica PowerCenter connection string, etc., or it can be left empty | Teradata |
| Included databases | List of databases to extract, separated by commas; leave blank to mark all schemas in the database | database1,database2 |
| Excluded databases | List of databases NOT to extract, separated by commas; leave blank to not restrict extractions | database1,database2 |
QueryGrid Database Links
Automatic Data Lineage supports references to tables and views in remote Teradata servers using the database.table@link_name syntax. The full table or view name (including database) must be used. The link name is matched to the mappings
described in the previous chapter using either the Dictionary
ID or Connection ID values.