Oracle Resource Configuration

Before you configure your scanner, make sure you meet the prerequisites. Read our guide on Oracle integration requirements to double-check.

Strong Security Performance Issue on Linux

There is a known issue with strong security on some Linux systems resulting in a connection reset error when connecting to Oracle via JDBC due to an empty entropy pool. To avoid this, set the following in <MANTA_DIR_HOME>/platform/bin/mantar.sh.

export JAVA_OPTS="$JAVA_OPTS -Djava.security.egd=file:/dev/./urandom"

Source System Properties

This configuration can be setup by creating a new connection on Admin UI / Connections tab or editing an existing connection in Admin UI / Connections / Databases / Oracle / specific connection. New connection can also be created via Manta Orchestration API.

Granularity of the IBM Automatic Data Lineage connection for Oracle is one Oracle service or instance. Use list of schemas to limit the scope of analysis as needed. Use of multiple connections against a single Oracle service or instance may lead to within-system lineage not be connected properly.

Property name

Description

Example

oracle.dictionary.id

Name of a resource representing this Oracle database known as the dictionary ID, used as an output subdirectory name for extracted DDL files and the database dictionary

OracleODS

oracle.url

JDBC connection string for the Oracle database. See Oracle documentation https://docs.oracle.com/en/database/oracle/oracle-database/21/jjdbc/data-sources-and-URLs.html#GUID-C4F2CA86-0F68-400C-95DA-30171C9FB8F0 for more details.

jdbc:oracle:thin:@<host>:<port>:<sid>
or
jdbc:oracle:thin:@//<host>:<port>/<service_name>
or
jdbc:oracle:thin:@ldap://<host>:<port>/<database>,cn=OracleContext,<ldap_dn_suffix>

or

jdbc:oracle:thin:@tcps://<host>:<port>/<service_name>[?ssl_server_cert_dn=<server cert DN>]
<server cert DN> looks like CN=testcert.oracle.com, O=Oracle Corporation,L=Redwood City,ST=California,C=US

or

jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCPS)(HOST=<host>)(PORT=<port>)))(CONNECT_DATA=(SERVICE_NAME=<service name>)) [(SECURITY=(SSL_SERVER_CERT_DN=<server cert DN>)) ])

oracle.username

Name of the user for connection to the Oracle database

manta_user

oracle.password

Password of the user for connection to the Oracle database

manta_password

oracle.extractedSchemas

List of schemas to extract, separated by commas, or leave blank to extract all schemas; supports regular expressions; the matching is case-sensitive

Schema1,Schema2,Schema3,Schema_.*

oracle.excludedSchemas

List of schemas to exclude from processing, separated by commas; if left blank, no schemas will be excluded

Schema_Test.*

oracle.excludedEntities

Specify the entities to exclude from extraction. The filter input is divided into two parts by a semicolon ( ; ). Each filter follows the same format of (Entity Types;)?Name Pattern.

1. Entity types (optional): The first part lists the types of entities to exclude. These can include table, view, materialized_view, package, package_body, type, type_body, procedure, function, trigger, sequence, synonym, or database_link. Multiple entity types should be separated by a | .

Note: Specifying package or type, etc. will automatically exclude also package_body or type_body, respectively. The reverse is also true, specifying package_body will automatically exclude package, etc.

2. Name pattern: is evaluated as a regular expression to match object name (regardless of schema that the object is in) or schema/objectname to limit the schema as well.

Both parts of the filter expression are case-insensitive.

You can specify multiple filters by separating them with a comma ( , ).

If a non-filtered entity (e.g., a view) points to a filtered entity (e.g., a table), the filtered entity will still appear as a deduced item.

  • temp.*: Excludes all objects whose name starts with temp, regardless of their type or schema

  • function;abc.*: Excludes functions whose name starts with abc, regardless of their schema

  • table|view;dw.*/.*,function|procedure;abc.*/def.*: Excludes every table and every view from schemas that start with dw and every function and procedure whose name starts with def from schemas that start with abc

oracle.extraction.method

Set to Agent:default when the desired extraction method is the default Manta Extractor Agent, set to Agent:{remote_agent_name} when a remote Agent is the desired extraction method, set to Git:{git.dictionary.id} when the Git ingest method is the desired extraction method. For more information on setting up a remote extractor Agent please refer to the Manta Flow Agent Configuration for Extraction documentation. For additional details on configuring a Git ingest method, please refer to the Manta Flow Agent Configuration for Extraction:Git Source documentation.

default

Git

agent

oracle.ddl.encoding

Encoding of automatically extracted Oracle DDL scripts. See Encodings for applicable values.

utf8

oracle.script.encoding

Encoding of manually provided PL/SQL scripts which are performed on this Oracle database instance. See Encodings for applicable values.

utf8

Common Properties

This configuration is common for all Oracle source systems and for all Oracle scenarios, and is configure in Admin UI / Configuration / CLI / Oracle / Oracle Common It can be overridden on individual connection level.

Property name

Description

Example

oracle.dictionary.dir

Directory with data dictionaries extracted from Oracle

${manta.dir.temp}/oracle

filepath.lowercase

Whether paths to files should be lowercase (false for case-sensitive file systems, true otherwise)

false
true

oracle.dll.output

Directory for automatically extracted Oracle DDL scripts (for the extraction phase)

${manta.dir.temp}/oracle/

${oracle.dictionary.id}/ddl

oracle.ddl.input

Directory with automatically extracted Oracle DDL scripts (for the analysis phase)

${oracle.dll.output}

oracle.script.input

Directory with manually provided PL/SQL scripts which are performed on this Oracle database instance (for the analysis phase)

${manta.dir.input}/oracle/

${oracle.dictionary.id}/

oracle.script.replace

Path to the CSV file with the replacements to be applied to the provided PL/SQL scripts; see Placeholder Replacement in Input Scripts for details about the replacement file format

${manta.dir.input}/oracle/
${oracle.dictionary.id}/replace.csv

oracle.script.replace.regex

Flag specifying whether replacements for PL/SQL scripts in the provided CSV file specified in oracle.script.replace should be interpreted as regular expressions (true) or simple text (false)

false
true

oracle.analyze.parallelCount

Number of parallel threads which will analyze DDL and PL/SQL

4

oracle.synonymProcessing

Toggles synonym visibility; set to READ_ONLY to show synonyms or TRANSPARENT to connect the lineage directly to the synonyms' targets

READ_ONLY
TRANSPARENT

oracle.dictionary.mappingFile

Path to automatically generated mappings for Oracle databases


${manta.dir.temp}/oracle/

oracleDictionaryMantaMapping.csv

oracle.dictionary.mappingManualFile

Path to mappings provided manually for Oracle databases


${manta.dir.scenario}/conf/

oracleDictionaryMantaMappingManual.csv

oracle.dynamicSql.enabled.ddl

Enables or disables dynamic SQL processing in DDL scripts

false
true

oracle.dynamicSql.enabled.plsql

Enables or disables dynamic SQL processing in PL/SQL scripts

false
true

oracle.expressionDescriptions.enabled.ddl

Enables or disables building transformation logic descriptions from SQL code in DDL scripts

false
true

oracle.expressionDescriptions.enabled.plsql

Enables or disables building transformation logic descriptions from SQL code in PL/SQL scripts

false
true

oracle.expressionDescriptions.enabled.queryService

Enables or disables building transformation logic descriptions from Oracle SQL code defined in other technologies

false
true

oracle.includeSchemaDependencies

Enables or disables the extraction of dependencies from non-included schemas (non-transitive)

Synonyms pointing into non-extracted schemas are only extracted if this flag is set to true

false
true

oracle.includeInvalidObjects

Enables or disables the extraction of invalid objects

false
true

oracle.instanceNameQuery

Query to get the name of the database that affects what is displayed in Manta and used in exports

select sys_context('userenv', 'instance_name') from dual

select name from v$database

custom

oracle.customInstanceNameQuery

When oracle.instanceNameQuery is set to custom, set the custom instance name query here; the query must return a single row with a column defining the desired name; see default queries in oracle.instanceNameQuery for inspiration

select customInstanceName from table

oracle.extractorScenario.inputReader

Set the extraction mode to scenarioReaderPrefetch (default) to use the prefetch extractor, to scenarioReader to use the legacy single thread extractor, or to scenarioReaderParallelBulk to use the bulk extractor

scenarioReaderPrefetch
scenarioReader
scenarioReaderParallelBulk

oracle.prefetchExtractor.numThreads

Number of parallel threads that will extract the metadata in the prefetch extractor

  • Impact: higher values allow more simultaneous connections to the source database, potentially consuming more CPU and more network bandwidth

  • Minimum: 1

  • Maximum: discuss with your database administrator

  • Recommended setting is 10

  • Anything over 20 is unlikely to further improve performance

10

oracle.prefetchExtractor.bufferSize

Size of the prefetch buffer used by the prefetch extractor

The bufferSize is the number of prefetched scripts, and the occupied RAM depends on the content of the scripts; for example, if the bufferSize is 100 and the average script size is 10KB, the total occupied RAM is 1 MB

  • Impact: higher values allow a bigger buffer, consuming more RAM in CLI

  • Minimum: 1

  • Maximum: depends on the available RAM

  • Recommended setting is 100–1000

100

oracle.bulkParallelExtractor.numThreads

Number of parallel threads that will extract the metadata in the bulk extractor

  • Impact: higher values allow more simultaneous connections to the source database, potentially consuming more CPU and more network bandwidth

  • Minimum: 1

  • Maximum: discuss with your database administrator

  • Recommended setting is 10

  • Anything over 20 is unlikely to further improve performance

10

oracle.analyze.retainUnusedResultSetColumns

Flag specifying whether the data lineage should include sub-query resultset columns that do not have any downstream lineage

By default, set to false

false
true

SSL for Encryption and Server Authentication

oracle.url must be set in the one of the following formats.

The Automatic Data Lineage trust store is used to validate the server’s certificate. If none of the trusted certificates contained in the trust store can be used to validate the server’s certificate, then the connection will fail with the exception message "Unable to find valid certification path to requested target".

Checking the Server’s Distinguished Name (Optional)

If the server is successfully authenticated (meaning its certificate is trusted), its DN can be checked. The expected DN is specified in the JDBC URL like in this example:
jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS=(PROTOCOL=tcps)(HOST=servername)(PORT=2484))(CONNECT_DATA=(SERVICE_NAME=service_name))(SECURITY=(SSL_SERVER_CERT_DN="CN=server_test,C=US")))

Manual Mapping Properties

In cases where an external connection to an Oracle database uses a different connection string or SID than the one automatically extracted by Automatic Data Lineage, it is necessary to manually configure the mappings for Oracle databases in the file ${oracle.dictionary.mappingManualFile}listed in the table above. This is also necessary for non-extracted schemas in the Automatic Data Lineage extraction phase and the complicated usage of database links. In order to configure manual mapping, copy a relevant line from the automatically generated file ${oracle.dictionary.mappingFile} and modify the Connection ID column with the external connection identifier.

Each mapping has its own row with the following parts separated by semicolons.

Property name Description Example
Global database name Database name unique to the whole system; can be fetched using the query SELECT * FROM global_name http://orcl.example.com
Database instance name Name of the instance, also called the SID orcl
Host name Name of the host where the database is running localhost
Host port Port on which the database listens 1521
Dictionary ID Name of a resource representing this Oracle database known as the dictionary ID, used as an output subdirectory name for extracted DDL files and the database dictionary OracleODS
Connection ID External Oracle connection ID in third-party tools, for example, as it appears in tnsnames.ora, Informatica PowerCenter connection string, etc., or it can be left empty if it is the same as the SID in the database instance name orcl
Included schemas List of schemas to be extracted from the database, separated by commas; leave blank to mark all schemas in the database schema1,schema2
Excluded schemas List of schemas NOT to be extracted from the database, separated by commas; leave blank to not restrict extractions schema1,schema2