MS SQL Resource Configuration
Source System Properties
This configuration can be setup by creating a new connection on Admin UI / Connections tab or editing an existing connection in Admin UI / Connections / Databases / MS SQL / specific connection. New connection can also be created via Manta Orchestration API.
Granularity of the IBM Automatic Data Lineage connection for MS SQL is one MS SQL server. Use filter on databases and schemas to limit the scope of analysis as needed. Use of multiple connections against a single MS SQL server may lead to within-system lineage not be connected properly.
|
Property name |
Description |
Example |
|---|---|---|
|
mssql.dictionary.id |
Name of a resource representing this MS SQL server known as dictionary ID, used as an output subdirectory name for extracted DDL files and the database dictionary |
SQLEXPRESS |
|
mssql.subdialect |
Sub-dialect of the target system in this connection
|
MSSQL |
|
mssql.url |
JDBC connection string for the MS SQL database |
jdbc:sqlserver://<host>:<port> |
|
mssql.username |
User name for connection to the MS SQL database |
manta_user |
|
mssql.password |
User password for connection to the MS SQL database |
manta_password |
|
mssql.extractedDbsSchemas |
List of databases and schemas to extract, separated by commas, which are to be provided in the format database/schema; each part is evaluated as a regular expression |
database1/sch1,database2/sch2,database3 |
|
mssql.excludedDbsSchemas |
List of databases and schemas to exclude from extraction, separated by commas |
database3/sch3,database3/sch4 |
|
mssql.excludedEntities |
Specify the entities to exclude from extraction. The filter input is divided into two parts by a semicolon character
Both parts of the filter expression are case-insensitive. You can specify multiple filters by separating them with a comma
If a non-filtered entity (e.g., a view) points to a filtered entity (e.g., a table), the filtered entity will still appear as a deduced item. Available as of R42.1 |
|
|
mssql.extraction.method |
Set to Agent:default when the desired extraction method is the default Manta Extractor Agent, set to Agent:{remote_agent_name} when a remote Agent is the desired extraction method, set to Git:{git.dictionary.id} when the Git ingest method is the desired extraction method. For more information on setting up a remote extractor Agent please refer to the Manta Flow Agent Configuration for Extraction documentation. For additional details on configuring a Git ingest method, please refer to the Manta Flow Agent Configuration for Extraction:Git Source documentation. |
default Git agent |
|
mssql.ddl.encoding |
Encoding of automatically extracted MS SQL DDL scripts. See Encodings for applicable values. |
utf8 |
|
mssql.script.encoding |
Encoding of manually provided TSQL scripts performed on this MS SQL database instance. See Encodings for applicable values. |
utf8 |
|
mssql.authenticationType |
Authentication type to use; see MS SQL Server Authentication Types for a description of the options |
SQL Server |
|
mssql.domain |
The Windows domain to authenticate to when using NTLM authentication |
example.com |
|
mssql.serverName |
User-set server name. Leave blank, to use the automatically extracted server name. Set to a custom unique server name to override the automatically extracted server name. For example, in Multi-AZ deployments for Amazon RDS for Microsoft SQL Server, the value should specify a "cluster name"; because (1) any of the individual SQL Server instances could be used during extraction (and therefore, the server name can change over time for Automatic Data Lineage) and (2) the individual server names are re-generated after each reboot with failover by AWS. |
SQLEXPRESS |
|
mssql.extractor.authentication.krb5Conf |
Kerberos realm configuration file; only valid for JavaKerberos authentication |
|
|
mssql.extractor.authentication.jaasConf |
JAAS configuration file that overrides the default settings of the Java Authentication and Authorization Service; only valid for JavaKerberos authentication |
|
The system sys schema that is present in every SQL Server database may only be extracted from the system master database. Its inclusion for extraction in any other database in the above configuration will be ignored.
Common Scanner Properties
This configuration is common for all MS SQL source systems and for all MS SQL scenarios, and is configure in Admin UI / Configuration / CLI / MSSQL / MSSQL Common. It can be overridden on individual connection level.
|
Property name |
Description |
Example |
|---|---|---|
|
mssql.dictionary.dir |
Directory with data dictionaries extracted from MS SQL |
${manta.dir.temp}/mssql |
|
filepath.lowercase |
Whether paths to files should be lowercase (false for case-sensitive file systems, true otherwise) |
false |
|
mssql.dll.output |
Directory for automatically extracted MS SQL DDL scripts (for the extraction phase) |
${manta.dir.temp}/mssql/ |
|
mssql.ddl.input |
Directory with automatically extracted MS SQL DDL scripts (for the analysis phase) |
${mssql.dll.output} |
|
mssql.script.input |
Directory with manually provided Transact-SQL scripts which are performed on this MS SQL database instance (for the analysis phase) |
${manta.dir.input}/mssql/ |
|
mssql.script.replace |
Path to the CSV file with the replacements to be applied to the provided Transact-SQL scripts; see Placeholder Replacement in Input Scripts for details about the replacement file format |
${manta.dir.input}/mssql/${mssql.dictionary.id}/replace.csv |
|
mssql.script.replace.regex |
Flag specifying whether replacements for Transact-SQL scripts in the provided CSV file specified in |
false |
|
mssql.analyze.parallelCount |
Number of parallel threads which will analyze DDL and Transact-SQL |
4 |
|
mssql.synonymProcessing |
Toggles synonym visibility; set to READ_ONLY to show synonyms or TRANSPARENT to connect the lineage directly to the synonyms' targets |
READ_ONLY |
|
mssql.dictionary.mappingFile |
Path to automatically generated mappings for MS SQL servers |
${manta.dir.temp}/mssql/ |
|
mssql.dictionary. |
Path to mappings provided manually for MS SQL servers |
${manta.dir.scenario}/conf/ |
|
mssql.connections.file |
Name of the file that contains linked server connection definitions in UTF-8 encoding This file only overrides definitions of linked servers extracted from MS SQL server. As such, it does not need to contain all the connections, just the ones where the user needs to change/add any of the connection attributes. For more information about this file and its format, see the section on Connection Definition Settings in Informatica PowerCenter Resource Configuration and get more details in (Manual) Connection Mappings Explained. |
linkedServerConnectionsConfiguration.prm |
|
mssql.connections.path |
Path to the linked server connection definitions base directory A path to the linked server connection definitions file will be built in the following format:
|
${manta.dir.input}/mssql |
|
mssql.expressionDescriptions.enabled.ddl |
Enables or disables building transformation logic descriptions from SQL code in DDL scripts |
false |
|
mssql.expressionDescriptions.enabled.script |
Enables or disables building transformation logic descriptions from SQL code in Transact-SQL scripts |
false |
|
mssql.expressionDescriptions.enabled.queryService |
Enables or disables building transformation logic descriptions from MS SQL SQL code defined in other technologies |
false |
|
mssql.extractor.enabled.encryptConnection |
Flag whether or not the connection with encryption is enabled. If true, the driver requests the use of TLS encryption with the server. If the server requires the client to support TLS encryption or if the server supports encryption, the driver will initiate the TLS certificate exchange. If the trustServerCertificate property is set to "true", the driver won't validate the TLS certificate. If the server isn't configured to support encryption, the driver will raise an error and terminate the connection The connection might still be encrypted even when this flag is set to false. However, the server certificate will not be validated. |
true |
|
mssql.extractor.hostNameInCertificate |
The driver will validate the TLS certificate's subject value using the provided host name in certificate. If the host name in the certificate is blank, the driver will use the server name to validate the server TLS certificate. If the server name is different from the DN (Domain Name) of the certificate, set the host name in the certificate to the server name. |
SQLSERVER |
|
mssql.extractor.enabled.trustServerCertificate |
If the trustServerCertificate property is set to "true", the driver won't validate the TLS certificate. This setting is common when allowing connections in test environments, such as when the SQL Server instance only has a self-signed certificate. |
false |
|
mssql.analyze.retainUnusedResultSetColumns |
Flag specifying whether the data lineage should include sub-query resultset columns that do not have any downstream lineage By default, set to false |
false |
Manual Mapping Properties
It is possible to manually configure mappings for MS SQL servers. This is necessary for non-extracted schemas in the Automatic Data Lineage extraction phase and complicated usage of database links.
Each mapping has its own row with the following parts separated by semicolons.
|
Property name |
Description |
Example |
|---|---|---|
|
Dictionary ID |
Name of a resource representing this MS SQL server known as dictionary ID, used as an output subdirectory name for extracted DDL files and the database dictionary |
SQLEXPRESS |
|
Server name |
Unique name of the server |
SQLSERVER |
|
Database instance name |
Name of the database instance |
SQLEXPRESS |
|
Host name |
Name of the host where the database is running |
localhost |
|
Host port |
Port on which the database listens |
1433 |
|
Connection ID |
External MS SQL connection ID in third-party tools; for example, as it appears in an SSIS package, Informatica PowerCenter connection string, etc., or it can be left empty if it is the same as the server name listed as the database instance name |
WIN-RR1232 |
|
Include filter |
List of databases and schemas to extract, separated by commas, which are to be provided in the format database/schema; each part is evaluated as a regular expression |
database1/sch1,database2/sch2,database3 |
|
Exclude filter |
List of databases and schemas to exclude from extraction, separated by commas |
database3/sch3,database3/sch4 |