Hive Integration Requirements

Prerequisites

The following are the prerequisites necessary for IBM Automatic Data Lineage to connect to this third-party system, which you may choose to do at your sole discretion. Note that while these are usually sufficient to connect to this third-party system, we cannot guarantee that the connection or integration will be successful since we have no control, liability, or responsibility for third-party products or services, including for their performance.

• Hive 2.3 or newer — Confirm exactly which version of Hive is being used by validating your big data platform vendor and version number or by checking directly by running the query select version() against Hive
• Hive JDBC driver provided by your Hadoop platform vendor
• Extract Hive metadata using Hive Server 2 via JDBC
• Kerberos authentication (optional)

Environment with Direct Access to Hive Server 2 via JDBC

• Database user rights
  • User with SELECT WITH GRANT OPTION rights for all tables in Hive databases that should be extracted (required for SHOW CREATE TABLE used by Automatic Data Lineage)
  • Automatic Data Lineage uses these statements to extract metadata: SHOW DATABASES, SHOW CREATE TABLE, SHOW TABLES IN, SHOW FUNCTIONS LIKE, and SHOW VIEWS IN
• Connection parameters for the JDBC connection
  • JDBC driver used
  • Name or IP address of Hive Server 2
  • Port on which Hive Server 2 listens
  • In the case of non-Kerberos authentication:
    • User name
    • User password
  • When using Kerberos authentication:
    • Principal of Hive Server 2 service
    • Optionally, a keytab and the principal that Automatic Data Lineage should use
  • Hive Server 2 must be accessible via network

Known Unsupported Features

Automatic Data Lineage does not support the following Hive features. This list includes all of the features that IBM is aware are unsupported, but it might not be comprehensive.

• Manta Flow or Manta Agent (depending on which component is using the Hive JDBC driver) running in a FIPS-compliant environment because Hive JDBC drivers are not compliant with FIPS
• Lineage through dynamically executed code through EXECUTE IMMEDIATE