Oracle lineage configuration

Connect to Oracle databases to scan metadata and display it on lineage. To import lineage metadata, create data source definition, a connection, and metadata import job.

Oracle is a multi-model database management system.

Supported Oracle versions

Oracle Database 11, 12, 18, 19, 21 (remote).

Processed metadata

The following Oracle metadata is processed and displayed on lineage:

  • PL/SQL scripts
  • Database dictionaries
  • Stored procedures
  • Functions
  • Packages
  • Triggers
  • Objects
  • Views
  • DB links
  • Exadata links

Limitations

The following limitations apply:

  • SQL*Plus scripts are not processed.
  • SQL*Loader scripts are not processed.
  • Highly dynamic SQL code that is executed by using the EXECUTE IMMEDIATE statement is not processed.
  • Synonyms are included in the data lineage visualization only if they are referenced in the SQL queries, procedures, or functions. If they are not referenced, they are not included in the visualization or the object list.

Prerequisite configuration

Before you import lineage metadata, ensure that the following prerequisites are met:

  • You must have the CONNECT and SELECT_CATALOG_ROLE privileges. An Oracle database administrator can grant these privileges. The SELECT_CATALOG_ROLE is required to execute the DBMS_METADATA.GET_DDL function, which is the only supported method for retrieving database metadata.
  • A firewall access must be opened between Oracle and IBM instances. An Oracle database administrator or the network team can open the firewall.
  • If you connect to Oracle 21.1 or earlier, and you use a connection in the read-only mode, install the following patch on the Oracle instance: 30858671 - 18C DBMS_METADATA.GET_DDL FAILED WITH ORA-16000 IN READ ONLY MODE.

Creating a metadata import asset

Data source connection

To connect to the data source from which you want to import lineage metadata, you need to select a data source definition and a connection. You can create them before you start creating the metadata import, or you can create them when you create and configure the metadata import asset.

Data source definition

Select Oracle as the data source type. The Database (SID) or Service name field is required.

Connection

For connection details, see Oracle connection.

Include and exclude lists

You can include or exclude assets up to the schema level. Each value is evaluated as a regular expression. Assets which are added later in the data source will also be included or excluded if they match the conditions specified in the lists. Example values:

  • mySchema: mySchema schema.
  • mySchema[1-5]: any schema with a name that starts with mySchema and ends with a digit between 1 and 5.

External inputs

If you use external Oracle PL/SQL scripts, you can add them in a .zip file as an external input. You can organize the structure of a .zip file as subfolders that represent schemas. Schemas are optional. If schemas are defined, the scripts are added under respective schemas in the selected catalog or project. The .zip file can have the following structure:

<schema_name>
   <script_name.sql>
<script_name.sql>
replace.csv

The replace.csv file contains placeholder replacements for the scripts that are added in the .zip file. For more information about the format, see Placeholder replacements.

Advanced import options

Extract extended attributes
You can extract extended attributes like primary key, unique and referential integrity constraints of columns. By default these attributes are extracted.
Extraction mode
You can decide which extraction mode to run for the imported metadata. You have the following options:
  • Prefetch: use it for relational databases.
  • Parallel bulk: use it for analytical processing engines.
  • Single-thread: use it to avoid parallelism and large queries during extraction. When you select this mode, performance might be low.
Transformation logic extraction
You can enable building transformation logic descriptions from SQL code in SQL scripts.
Dynamic SQL
Enable dynamic SQL processing in the DDL scripts. Dynamic SQL is included in lineage when the script contains enough static information to reconstruct the generated SQL statement. It is applicable to cases where the SQL is built mainly from literals or simple string concatenations, and the full structure of the statement is visible in the script. Dynamic SQL cannot be included in lineage when key elements, such as schema names, table names, or conditions, are obtained at runtime from the database. In these cases, the script does not contain the information which is required to determine lineage, and additional user input is needed.

Learn more