IBM IGC: Quick Start Guide / Configuration Checklist
IBM IGC Prerequisites
For more information about the necessary prerequisites, see Manta Flow IBM IGC Requirements.
Manta Preparation and Steps
Certificate Setup for the HTTPS Connection to IBM IGC
IBM IGC uses a secure HTTPS connection for the REST interface by default. In order to establish the secure connection from Manta Flow to IBM IGC, IGC's signer certificate needs to be added to Automatic Data Lineage list oftrusted certificates.
- Provide the trusted signer certificate you use for IGC or extract a (default) signer certificate from the WebSphere Application Server (WAS) Administration Console to a file by navigating through the WAS Console as follows: WAS Admin Console > Security > SSL certificate and key management > Key stores and certificates > NodeDefaultTrustStore > Signer certificates
{IS_install_path}/wlp/usr/servers/iis/resources/security/iis-server-keystore.p12 file. For more information, see SSL certificates for WebSphere Application Server Liberty Profile.
- Load the extracted certificate into the Automatic Data Lineage JKS by using Java Keytool or another keystore management tool. There is an empty keystore available in
<MANTA_DIR_HOME>/scenarios/manta-dataflow-cli/etc/igctruststore. The default password for the keystore isibmigc.
Manta Admin UI provides a keytool in the UI for adding SSL certificates.
The SSL certificates can be uploaded directly into Admin UI if you have them available on your local machine, or the certificates can be retrieved directly from the IGC Server by providing Automatic Data Lineage with a link to the IGC page.
Once these steps have been successfully completed, you can configure the Automatic Data Lineage to IBM IGC Export connection via Manta Admin UI.
IBM IGC Connection Properties
This configuration is common for all source systems and is stored in the
igcExport.properties file. These configurations can be updated and maintained from within Manta Admin UI and are located in the
Integrations section under the Configuration tab.
The following is a link to the page. Replace <manta-server-url> with your Automatic Data Lineage server URL.
<manta-server-url>/app/#/platform/configuration/group/Integrations/category/IGC/resource/IGC%20Export%20Common
Enter the connection details for all the required fields and save the configuration.
IBM IGC Resource Mapping Configuration
In IBM Information Governance Catalog all metadata from data resources like RDBMSs is assigned to a host and database. Manta Flow has to know this structure to be able to connect the newly created metadata with an IGC native resource. This mapping
is covered by a simple configuration that consists of the requested names of the host and database in IGC and a definition of the database objects that should be in this resource. The definition of database elements is slightly different for each
SQL dialect. The location for configuring these definitions is
<manta-server-url>/app/#/platform/configuration/group/Integrations/category/IGC/resource/${<technology>}%20IGC%20Mapping where ${<technology>} represents the exported technology.
Enter the mapping details for each resource you are planning to export to IGC and save the configuration.
The IGC Host and IGC Database used in the configuration relate to the highlighted elements in your IGC Implemented Data Resources as follows.
Each resource technology has its own resource mapping configuration. Here is an example of a mapping for an MS SQL resource.
Microsoft SQL Server
-
File name housing the mapping configuration (also available in Admin UI):
mssqlIgcMantaMapping.csv -
Format:
IGC Host;IGC Database;Host;Include;Exclude-
IGC Host—the host name in IGC -
IGC Database—the database name in IGC -
Host—SQL server name and instance name (this is simply the name of the server as displayed in the Manta Dataflow Viewer), separated by a double backslash (not all server names displayed in Automatic Data Lineage will include the\\characters; only include these when they are in the displayed name in Automatic Data Lineage), that list which elements should be in this resource -
Include—list of databases and schemas, separated by commas, which should be in this resource; databases and schemas are to be entered using the format database/schema; each part is evaluated as a regular expression -
Exclude—list of databases and schemas, separated by commas, which should not be in this resource; databases and schemas are to be entered using the format database/schema; each part is evaluated as a regular expression
-
-
Example:
IGC Host;IGC Database;Host;Include;Exclude 192.168.25.1;MSSQLMart;SQL08;mart/.*;mart/test_.*
IBM IGC Preparation and Steps
Importing the Manta Meta Model to IGC
Manta Flow uses its own meta model in IBM Information Governance Catalog. This model must be uploaded prior to any exports being executed.
This can be done via an IBM IGC REST API call, which can conveniently be done through the IBM IGC Swagger page — the URL for this operation is <Infosphere_url>/ibm/iis/igc-rest-explorer/#!/bundles/registerBundle. The Manta Meta
Model itself is stored in the directory <MANTA_DIR_HOME>/scenarios/manta-dataflow-cli/model.
Updating WebSphere API Limits for Lineage Upload
The default limits configured for the WebSphere Application Services API may not be sufficient for loading data lineage into IGC. If you need to increase the maximum memory allocation or number of objects kept in memory. To incerease the limits, see Prepare Information Governance Catalog for use with IBM Industry Models.
It is also necessary to increase the session timeout as described in Changing timeout values to allow import of large files to InfoSphere Business Glossary or to InfoSphere Information Governance Catalog.
Updating the Manta IGC Export Common Payload Size
An alternative/parallel approach to combating the limits configured for the WebSphere Application Services API is to reduce the Manta Export payload size by lowering the asset count per batch value. By default, this value is set to
10000. You can reduce it to a lower number by navigating to the Export Settings section in the IGC Export Common page: <manta-server-url>/manta-admin-gui/app/#/platform/configuration/group/Integrations/category/IGC/resource/IGC%20Export%20Common and updating the field value.
Manta Flow IBM IGC Export Execution
To export metadata from the Manta Flow metadata repository and upload it to IBM IGC:
-
Once all the necessary metadata is in Manta Flow Server, run the
Run Export IGCworkflow; otherwise, execute the whole extraction, analysis, and export process by running theRun IGCscenario. -
After the export is completed, the temporary output files used for upload are stored in the
${manta.dir.output}/<technology>/${<technology>.dictionary.id}/igcfolder, where<technology>represents the exported technology.
Run Export IGC workflow without targeted scans enabled, only the dictionary mapping information for the most recent extraction for
the technology is available for export into IGC. This might cause errors when exporting data that was gathered from resources scanned prior to the most recent extraction. An example of an error message you might run into in this situation is There is no mapping for the....
To allow exporting data that was gathered over time, rather than the most recently extracted data set, you can either:- Enable targeted scanning.
- Each time you want to export data that was captured earlier, run
<technology>DictionaryMappingScenario for the selected connections before you run the Run Export IGC scenario. As a result, a new dictionary mapping is generated to use with the IGC communication.