Oracle Resource Configuration
Before you configure your scanner, make sure you meet the prerequisites. Read our guide on Oracle integration requirements to double-check.
Strong Security Performance Issue on Linux
There is a known issue with strong security on some Linux systems resulting in a connection reset error when connecting to Oracle via JDBC due to an empty entropy pool. To avoid this, set the following in
<MANTA_DIR_HOME>/platform/bin/mantar.sh.
export JAVA_OPTS="$JAVA_OPTS -Djava.security.egd=file:/dev/./urandom"
Source System Properties
This configuration can be setup by creating a new connection on Admin UI / Connections tab or editing an existing connection in Admin UI / Connections / Databases / Oracle / specific connection. New connection can also be created via Manta Orchestration API.
Granularity of the IBM Automatic Data Lineage connection for Oracle is one Oracle service or instance. Use list of schemas to limit the scope of analysis as needed. Use of multiple connections against a single Oracle service or instance may lead to within-system lineage not be connected properly.
|
Property name |
Description |
Example |
|---|---|---|
|
oracle.dictionary.id |
Name of a resource representing this Oracle database known as the dictionary ID, used as an output subdirectory name for extracted DDL files and the database dictionary |
OracleODS |
|
oracle.url |
JDBC connection string for the Oracle database. See Oracle documentation https://docs.oracle.com/en/database/oracle/oracle-database/21/jjdbc/data-sources-and-URLs.html#GUID-C4F2CA86-0F68-400C-95DA-30171C9FB8F0 for more details. |
or
or
|
|
oracle.username |
Name of the user for connection to the Oracle database |
manta_user |
|
oracle.password |
Password of the user for connection to the Oracle database |
manta_password |
|
oracle.extractedSchemas |
List of schemas to extract, separated by commas, or leave blank to extract all schemas; supports regular expressions; the matching is case-sensitive |
Schema1,Schema2,Schema3,Schema_.* |
|
oracle.excludedSchemas |
List of schemas to exclude from processing, separated by commas; if left blank, no schemas will be excluded |
Schema_Test.* |
|
oracle.excludedEntities |
Specify the entities to exclude from extraction. The filter input is divided into two parts by a semicolon ( ; ). Each filter follows the same format of
1. Entity types (optional): The first part lists the types of entities to exclude. These can include Note: Specifying 2. Name pattern: is evaluated as a regular expression to match object name (regardless of schema that the object is in) or schema/objectname to limit the schema as well. Both parts of the filter expression are case-insensitive. You can specify multiple filters by separating them with a comma ( , ). If a non-filtered entity (e.g., a view) points to a filtered entity (e.g., a table), the filtered entity will still appear as a deduced item. |
|
|
oracle.extraction.method |
Set to Agent:default when the desired extraction method is the default Manta Extractor Agent, set to Agent:{remote_agent_name} when a remote Agent is the desired extraction method, set to Git:{git.dictionary.id} when the Git ingest method is the desired extraction method. For more information on setting up a remote extractor Agent please refer to the Manta Flow Agent Configuration for Extraction documentation. For additional details on configuring a Git ingest method, please refer to the Manta Flow Agent Configuration for Extraction:Git Source documentation. |
default Git agent |
|
oracle.ddl.encoding |
Encoding of automatically extracted Oracle DDL scripts. See Encodings for applicable values. |
utf8 |
|
oracle.script.encoding |
Encoding of manually provided PL/SQL scripts which are performed on this Oracle database instance. See Encodings for applicable values. |
utf8 |
Common Properties
This configuration is common for all Oracle source systems and for all Oracle scenarios, and is configure in Admin UI / Configuration / CLI / Oracle / Oracle Common It can be overridden on individual connection level.
|
Property name |
Description |
Example |
|---|---|---|
|
oracle.dictionary.dir |
Directory with data dictionaries extracted from Oracle |
${manta.dir.temp}/oracle |
|
filepath.lowercase |
Whether paths to files should be lowercase (false for case-sensitive file systems, true otherwise) |
false |
|
oracle.dll.output |
Directory for automatically extracted Oracle DDL scripts (for the extraction phase) |
${manta.dir.temp}/oracle/ ${oracle.dictionary.id}/ddl |
|
oracle.ddl.input |
Directory with automatically extracted Oracle DDL scripts (for the analysis phase) |
${oracle.dll.output} |
|
oracle.script.input |
Directory with manually provided PL/SQL scripts which are performed on this Oracle database instance (for the analysis phase) |
${manta.dir.input}/oracle/ ${oracle.dictionary.id}/ |
|
oracle.script.replace |
Path to the CSV file with the replacements to be applied to the provided PL/SQL scripts; see Placeholder Replacement in Input Scripts for details about the replacement file format |
${manta.dir.input}/oracle/ |
|
oracle.script.replace.regex |
Flag specifying whether replacements for PL/SQL scripts in the provided CSV file specified in |
false |
|
oracle.analyze.parallelCount |
Number of parallel threads which will analyze DDL and PL/SQL |
4 |
|
oracle.synonymProcessing |
Toggles synonym visibility; set to READ_ONLY to show synonyms or TRANSPARENT to connect the lineage directly to the synonyms' targets |
READ_ONLY |
|
oracle.dictionary.mappingFile |
Path to automatically generated mappings for Oracle databases
|
${manta.dir.temp}/oracle/ oracleDictionaryMantaMapping.csv |
|
oracle.dictionary.mappingManualFile |
Path to mappings provided manually for Oracle databases
|
${manta.dir.scenario}/conf/ oracleDictionaryMantaMappingManual.csv |
|
oracle.dynamicSql.enabled.ddl |
Enables or disables dynamic SQL processing in DDL scripts |
false |
|
oracle.dynamicSql.enabled.plsql |
Enables or disables dynamic SQL processing in PL/SQL scripts |
false |
|
oracle.expressionDescriptions.enabled.ddl |
Enables or disables building transformation logic descriptions from SQL code in DDL scripts |
false |
|
oracle.expressionDescriptions.enabled.plsql |
Enables or disables building transformation logic descriptions from SQL code in PL/SQL scripts |
false |
|
oracle.expressionDescriptions.enabled.queryService |
Enables or disables building transformation logic descriptions from Oracle SQL code defined in other technologies |
false |
|
oracle.includeSchemaDependencies |
Enables or disables the extraction of dependencies from non-included schemas (non-transitive) Synonyms pointing into non-extracted schemas are only extracted if this flag is set to |
false |
|
oracle.includeInvalidObjects |
Enables or disables the extraction of invalid objects |
false |
|
oracle.instanceNameQuery |
Query to get the name of the database that affects what is displayed in Manta and used in exports |
custom |
|
oracle.customInstanceNameQuery |
When |
select customInstanceName from table |
|
oracle.extractorScenario.inputReader |
Set the extraction mode to
|
scenarioReaderPrefetch |
|
oracle.prefetchExtractor.numThreads |
Number of parallel threads that will extract the metadata in the prefetch extractor
|
10 |
|
oracle.prefetchExtractor.bufferSize |
Size of the prefetch buffer used by the prefetch extractor The
|
100 |
|
oracle.bulkParallelExtractor.numThreads |
Number of parallel threads that will extract the metadata in the bulk extractor
|
10 |
|
oracle.analyze.retainUnusedResultSetColumns |
Flag specifying whether the data lineage should include sub-query resultset columns that do not have any downstream lineage By default, set to false |
false |
SSL for Encryption and Server Authentication
oracle.url must be set in the one of the following formats.
-
As of Oracle 19c use
jdbc:oracle:thin:@tcps://<host>:<port>/<service_name>[?ssl_server_cert_dn=<server cert DN>] -
Or
jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCPS)(HOST=<host>)(PORT=<port>)))(CONNECT_DATA=(SERVICE_NAME=<service name>)) [(SECURITY=(SSL_SERVER_CERT_DN=<server cert DN>)) ])
The Automatic Data Lineage trust store is used to validate the server’s certificate. If none of the trusted certificates contained in the trust store can be used to validate the server’s certificate, then the connection will fail with the exception message "Unable to find valid certification path to requested target".
Checking the Server’s Distinguished Name (Optional)
If the server is successfully authenticated (meaning its certificate is trusted), its DN can be checked. The expected DN is specified in the JDBC URL like in this example:jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS=(PROTOCOL=tcps)(HOST=servername)(PORT=2484))(CONNECT_DATA=(SERVICE_NAME=service_name))(SECURITY=(SSL_SERVER_CERT_DN="CN=server_test,C=US")))
Manual Mapping Properties
In cases where an external connection to an Oracle database uses a different connection string or SID than the one automatically extracted by Automatic Data Lineage, it is necessary to manually configure the mappings for Oracle databases in the
file ${oracle.dictionary.mappingManualFile}listed in the table above. This is also necessary for non-extracted schemas in the Automatic Data Lineage extraction phase and the complicated usage of database links. In order to configure
manual mapping, copy a relevant line from the automatically generated file ${oracle.dictionary.mappingFile} and modify the Connection ID column with the external connection identifier.
Each mapping has its own row with the following parts separated by semicolons.
| Property name | Description | Example |
|---|---|---|
| Global database name | Database name unique to the whole system; can be fetched using the query SELECT * FROM global_name | http://orcl.example.com |
| Database instance name | Name of the instance, also called the SID | orcl |
| Host name | Name of the host where the database is running | localhost |
| Host port | Port on which the database listens | 1521 |
| Dictionary ID | Name of a resource representing this Oracle database known as the dictionary ID, used as an output subdirectory name for extracted DDL files and the database dictionary | OracleODS |
| Connection ID | External Oracle connection ID in third-party tools, for example, as it appears in tnsnames.ora, Informatica PowerCenter connection string, etc., or it can be left empty if it is the same as the SID in the database instance name | orcl |
| Included schemas | List of schemas to be extracted from the database, separated by commas; leave blank to mark all schemas in the database | schema1,schema2 |
| Excluded schemas | List of schemas NOT to be extracted from the database, separated by commas; leave blank to not restrict extractions | schema1,schema2 |