IBM Automatic Data Lineage Configuration

The configuration of Automatic Data Lineage in the Admin UI consists of two tabs, namely Connections and Configuration. You can see what each tab is for below.

Tab uses:

• Connections tab

  • Configure connections to source systems that should be analyzed by Manta data lineage scanners (Manta Flow CLI)

• Configuration tab

  • Configure the general behavior of Manta data lineage scanners (Manta Flow CLI)

  • Configure the general behavior of the Manta data lineage repository (Manta Flow Server)

  • Configure the general behavior of Manta data lineage visualization (Manta Flow Server)

  • Configure the authentication and authorization of Manta Flow Server and its graphical (data lineage visualization) and programming (Manta Repository API, export/import features) APIs.

  • Set a new license for Manta and see the license statistics

Configuration Types

There are two configuration types that can be found in the Manta Admin UI: properties configuration and tabular configuration.

Properties Configuration

Properties configuration represents a configurable element as a list of properties. The name and general description of the configurable element may be found at the top of the viewer. The required properties, which may not be empty, have a red asterisk before the property name. The configuration may be in one of two modes: read or edit.

Basic/Advanced

You can choose between the two view modes: Basic and Advanced. The basic mode shows only basic configuration properties while the advanced mode shows basic configuration properties, hidden configuration properties, and advanced configuration properties (parent configuration properties, which may be overridden and present only if the configurable element has a parent configuration).

The behavior of the advanced mode also depends on the configuration mode: read or edit. In read mode, only overridden properties will be shown.

In edit mode, all properties, overridden and default, will be shown. To make advanced configuration properties editable and override values from the parent configuration, use the switch at the beginning of the Advanced Configuration section. The revert symbol next to the property value in the Advanced Configuration part serves two purposes. It is an indicator of an overridden property and a button for reverting the

default value of the property.

Property Description

Each defined property in Manta has a description that can be revealed by hovering over the info icon next to the property.

Some radio button and checkbox property values also have descriptions that are displayed by hovering over the property value.

Edit

To enter edit mode to change the connection configuration, click the blue Edit button in the upper-right corner. Once you enter edit mode, you will be able to edit all editable properties. Read-only properties will remain uneditable.

The following types of properties can be found in the property configuration:

• Text property — a property with one string or number value

• Multi-field text property — a property with any number of string or number values

• Radio button property — a property with one checked value from a predefined list of possible values

• Checkbox property — a property with any number of checked values from a predefined list of possible values

• Select property — a property with one selected value from a predefined list of possible values

• Keystore editor — a special property type that allows you to manage keystores and/or truststores. (The following text mentions only keystores to improve readability, the documentation applies to both.)

If the password or path to a keystore is changed, the configuration should be saved before the keystore setting is opened; otherwise, the old property values will be used when editing the keystore.

After clicking on the Edit button, all private keys and/or certificates in the store are listed. It’s possible to delete and add new certificates to the store in this window.

Each certificate has additional information, which can be displayed after clicking on the arrow.

The keystore can be recreated with a new password and path. If the keystore is recreated, all certificates will be lost and the old keystore will be deleted.

It’s possible to add new entries to the keystore by either adding entries from another keystore or downloading a certificate. A certificate can be downloaded from a website or from supported technologies directly by providing URL.

When entries are extracted from the keystore or a website, then we can choose which entries to add and change their default alias to a custom alias.

Note that Manta has no means of validating the origin of the certificate. It is the responsibility of the user importing the certificate to make sure that the certificate being imported is the one desired and that Manta should trust it.

Each entry is checked to determine if it can be added to the keystore. If the configuration screen contains a truststore component, only public certificates can be uploaded. If the configuration screen contains a keystore component, only private keys can be uploaded. In some specific cases (e.g., Qlik Sense connection definition), the components can be mixed — accepting both public certificates and private keys. If the entry cannot be added, then a red triangle will be displayed on the row with the entry and it won’t be possible to select it for import.

Save

When you click the Save button on the top panel, the property and configuration validation will be initialized, and if it’s successful, then the configuration will be saved. You cannot undo the configuration save.

Validation

There are two types of validations which are provided:

• Property validation

• Configuration validation

Property Validation

Most of the properties have validation rules. In case of non-compliance with the rules, a message about failed validation is shown under the validated property. Property validation is initialized when a user interacts with a property or saves a configuration. A configuration cannot be saved until all the property validation rules are fulfilled.

Configuration Validation

Most configurations are automatically validated before saving or upon explicit request by the user by clicking on Validate button. If the validation is successful, you will be notified; otherwise, a dialog window with information about the failed validation opens.

If Manta is able to automatically extract metadata from a particular technology for flow analysis, it is possible to validate a connection to the source system by clicking on the Validate button.

Some validations may take up to a few minutes to complete, and it’s possible to abort a running validation. The validation steps that have not been run yet will be skipped. The aborted validation is shown as failed, and it's necessary to confirm that the configuration is in the desired state to save it. It is not good practice to skip validations, as it may cause problems during extraction, so this feature should only be used in rare situations.

Custom Properties

All custom properties that aren’t provided by Manta are displayed in the Custom Settings property group. Such properties can be edited, but you cannot add new ones (which should not be needed anyway). These are typically leftovers from older versions of Manta where the property was used, but it is not used anymore. These properties can be removed by editing the underlying property file directly in the file system. Such properties should not cause issues as they should not be used anymore.

Tabular Configuration

Tabular configuration is represented by a table. It’s possible to add, delete, and edit lines. There are two different types of tables that are used to represent different types of configurations, which are the simple table and the manual override table.

If there are many lines in a table, the lines will be paged so you can choose how many lines you want to see on each page. Each column has a description, which will be shown when you hover over the info icon next to a column name.

Delete Lines

A line from the table may be deleted in two ways.

• Use the recycle bin icon for the line you want to delete. (No recycle bin icon is displayed for lines that cannot be deleted.)

• Select multiple lines using the checkbox at the beginning of each line (the checkbox is grayed out for lines that cannot be deleted) and click on the recycle bin next to the search box.

Edit Existing Lines

To edit an existing line, click on the pencil at the end of the line. (If the line is not editable, there will not be a pencil.)

After clicking the edit button, the line configuration is represented by the properties configuration.

Search

You can search all columns in the table using the search box in the upper-right corner.

Simple Tables

A simple table is a typical table without any modifications.

Add New Lines

New lines can be added to the simple table using the Add Line button in the header.

After clicking on the button, a form that can be used to add a new line is shown.

Copy Lines

Another way to add new lines is to copy an existing line using the copy line button at the end of the line that you want to copy.

After clicking the button, a form that can be used to add a new line is displayed. The form will be prefilled with the values from the line that you chose to copy.

Manual Override Table

The manual override table differs in how new lines are added to it. A new line is added by copying an existing line and overriding the values from it.

Add New Lines

New lines can be added to the manual override table using the Manual Override button at the end of the line you want to copy and override. New lines can only be added to the manual override table by overriding existing lines.

After clicking the button, a form that can be used to add a new line is displayed. By default, only the basic properties to be overridden are shown. So, if you want to override any other properties, click on the Full Override button.

Connections Tab

The Connections tab contains existing connections to source systems that have already been created. The tab allows you to create new connections and configure existing ones.

Connections Menu

The connections menu contains connections to source systems. Each item in the menu represents a separate connection. If a connection item is crossed out and has a red crossed-out circle next to it, then it’s disabled and won’t be analyzed; otherwise, it’s enabled and will be analyzed during Manta Flow analysis. Connections are divided into different groups by technology type and technology. When a connection is selected, it is highlighted in light red. The path to the selected item is lined with a red technology group and technology names so it’s easy to find the location of the element that is currently selected, even if all groups are collapsed. Each group can be collapsed as well as the whole menu.

Connection Viewer and Editor

After selecting a connection from the connection menu, the connection configuration viewer appears in read-only mode, displaying the configuration without the option to change it. The connection configuration is represented by the properties configuration.

Enable/Disable Connections

A connection can be in two states.

• Enabled — taken into account during Manta Flow analysis

• Disabled — ignored during Manta Flow analysis

The connection state can be changed using the switch next to the connection name, which also serves as a connection state indicator.

The second indicator is the connection item in the connection menu. When a connection item in the menu is crossed out and has a red crossed-out circle next to it, it’s disabled; otherwise, it’s enabled.

Delete Connections

When you click the Delete button, the Delete Connection? dialog box appears. You can still cancel the deletion at this time, but once you click the Delete button in the dialog box, the deletion cannot be undone.

Create New Connections

To create a new connection, you can use the + Add Connection button in the connections menu or in the center of the Connections page under the greeting.

New Connection Dialog Box

After clicking on the + Add Connection button, the Configure New Connection dialog box will appear and you can choose which technology you want to create a new connection for. The technologies are divided into technology groups. Each item in these groups contains a technology icon and short and full technology names. At the top of the dialog box, there is a search field that can be used to search for the necessary technology.

The new connection configuration is represented by the properties configuration.

Configuration Tab

The Configuration tab allows us to configure the general behavior of Manta, the authentication and authorization of the Manta Flow Server, and the graphical and programming APIs as well as set a new license for Manta and see the license statistics.

Configuration Menu

The configuration menu contains the common Manta elements that can be configured.

Responsibilities of the menu elements:

• CLI

  • Common

    • Common Config — common configuration for all source systems

    • File Path Mapping — mapping of the filesystem paths to ensure that the same file can be correctly referenced from different technologies; see Filesystem Resource Configuration for details

• Scanners

  • <Source system name>

    • <Source system name> Common — common configuration for the source system

    • <Source system name> Manual — manual mapping configuration for the source system servers (only present if Manta provides automatic export for the source system)

• Server

  • Common

    • Viewer Limits — configuration of the response limits for client queries

    • Repository Configuration — general repository configuration

    • System properties — general system configuration

  • Security

    • Security — configuration of security modes

    • LDAP — LDAP authentication and authorization—configuration of the connection to the LDAP server

    • SAML — configuration of the SSO integration

    • LDAP Mapping — mapping the LDAP groups to the Manta roles

    • SAML Mapping — mapping of the user or group identifiers retrieved from the SAML response to Manta application roles

• Integrations

  • Collibra

    • Collibra Export — common configuration for export to Collibra DGC and configuration of the connection to Collibra DGC

    • Collibra Entity Types — common configuration for export to Collibra DGC and configuration of the connection to Collibra DGC

    • Collibra Export Filter — parts of the Manta repository will be exported to Collibra

    • Collibra Mapping — mapping of the Manta database hierarchy to the Collibra hierarchy

    • Filesystem Mapping — mapping of the Manta filesystem hierarchy to the Collibra hierarchy

  • IEDC

    • <Source system name> IEDC Export Common — common configuration for the export of source system resources to IEDC

    • <Source system name> IEDC Export Master — master configuration for the export of source system resources to IEDC

    • <Source system name> IEDC Mapping — configuration determines which parts of the source system resource will be exported to Informatica EDC

  • IGC

    • <Source system name> IGC Export Common — common configuration for the export of source system resources to IGC

    • <Source system name> IGC Mapping — configuration determines which parts of the Manta source system resource will be exported to IBM Information Governance Catalog

  • Open Manta Integration Export

    • <Source system name> Open Manta Integration Export Common — common configuration for the Open Manta integration export of source system resources

    • <Source system name> Open Manta Integration Export Master — master configuration for the Open Manta integration export of source system resources

  • License — see the current Manta license, set a new license for Manta, and see the license statistics

Configuration Viewer and Editor

In the Configuration tab, two types of configurations are used: properties configuration and tabular configuration.

License

On the License Management page, you can see the Current License Information and License Statistics and you can set a new license.

Apply a New License

To apply a new license, you should press the Upload New License button, which will open a dialog box where you can drop a license file, or use the Upload License button to choose a file from your file system.

After uploading a license to Manta, you will see information about the license and if the license is valid. If it is, then you can press the Apply License button to apply the license.

If there are any problems with the license, you will see why the validation failed. Contact IBM Support if this happens.

Maintenance

The Maintenance page contains a directory of links to individual Manta components/functions that are useful for Manta administration.

Related links:

• Back Up Manta Repository — create a backup of the Manta Repository content

• Restore Manta Repository — restore the Manta Repository contents from a backup file

• Manta Repository API documentation — open the Manta Repository API documentation and test page (Swagger)

• Manta Repository Usage Statistics — export Manta usage statistics

• Manta Administration API documentation — open the Manta Administration API documentation and test page (Swagger)