IBM Fluid Query 1.7.2

JDBC drivers for predefined data connectors

The data connector feature requires you to obtain JDBC drivers from your service providers and install them on the NPS® appliance.

You only need to obtain drivers for the service provider that you are plan to use.

The following table lists the supported service providers, the location on the NPS appliance where the JDBC drivers must be stored, and the required JDBC driver files. You only need to obtain drivers for the service provider that you are plan to use. The sample file list and the file revision numbers could change depending on the service provider and driver version that you use, as well as with updates to those software releases or JAR files.

The NPS path column shows the directory under the /nz/export/ae/products/fluidquery/ where you must save the JDBC files for each service provider that you use. Make sure that you store the JDBC files in the correct locations on the NPS appliance.

Table 1. List of JDBC drivers for the supported service providers
Provider Service NPS path Sample file list
Oracle Oracle ./libs/oracle/oracle/ Drivers are available on the Oracle website.
Apache Spark SQL

./libs/ibm/sparksql/
./libs/cloudera/sparksql/
./libs/horton/sparksql/

As listed in this table for your provider's Hive.
IBM® IBM PureData® System for Analytics ./libs/ibm/pda/ nzjdbc3.jar
DB2® ./libs/ibm/db2/ db2jcc.jar
Db2 Warehouse ./libs/ibm/dashdb/ db2jcc.jar
PureData System for Operational Analytics ./libs/ibm/pdoa/ db2jcc.jar
IBM BigInsights® 4.1 Spark SQL ./libs/ibm/sparksql/ spark-assembly-1.4.1_IBM_2-hadoop2.7.1-IBM-8.jar located in /usr/iop/current/spark-client/lib/
IBM BigInsights 4 Hive 2 ./libs/ibm/hive

hive-jdbc-0.14.0_IBM_8-standalone.jar
hadoop-common-2.6.0-IBM-7.jar
log4j.properties

BigSQL ./libs/ibm/bigsql db2jcc.jar
IBM BigInsights 3 Hive 2 ./libs/ibm/hive/

commons-configuration-1.6.jar
commons-logging-1.1.1.jar
hadoop-core-2.2.0-mr1.jar
hive-exec-0.12.0.jar
hive-jdbc-0.12.0.jar
hive-metastore-0.12.0.jar
hive-service-0.12.0.jar
hive-shims-0.12.0.jar
httpclient-4.2.5.jar
httpcore-4.2.4.jar
libfb303-0.9.0.jar
libthrift-0.9.0.jar
log4j-1.2.17.jar
slf4j-api-1.6.1.jar
slf4j-log4j12-1.6.1.jar

BigSQL ./libs/ibm/bigsql/ db2jcc.jar
BigSQLv1 ./libs/ibm/bigsqlv1/ bigsql-jdbc-driver.jar
Cloudera Impala/Hive2 ./libs/cloudera/hive/./libs/cloudera/impala You can download drivers from the Cloudera website or download the following drivers from a Cloudera installation. See Obtaining Cloudera JDBC drivers later in this topic.

commons-collections.jar
commons-configuration.jar
commons-lang.jar
commons-logging-1.0.4.jar
guava.jar
hadoop-annotations-2.0.0-cdh4.6.0.jar
hadoop-auth-2.0.0-cdh4.6.0.jar
hadoop-common-2.0.0-cdh4.6.0.jar
hadoop-mapreduce-client-core.jar
hive-exec-0.10.0-cdh4.6.0.jar
hive-jdbc-0.10.0-cdh4.6.0.jar
hive-metastore-0.10.0-cdh4.6.0.jar
hive-service-0.10.0-cdh4.6.0.jar
hive-shims-0.10.0-cdh4.6.0.jar
httpclient-4.2.5.jar
httpcore-4.2.5.jar
libfb303-0.9.0.jar
log4j-1.2.16.jar
slf4j-api-1.6.4.jar
slf4j-log4j12-1.6.1.jar

Hortonworks Hive2 ./libs/horton/hive/
For Hortonworks Data Platform version 2.3:

commons-codec-1.4.jar
commons-collections-3.2.1.jar
commons-configuration-1.6.jar
commons-logging-1.1.3.jar
hadoop-auth-2.4.0.2.1.7.0-784.jar
hadoop-common-2.4.0.2.1.7.0-784.jar
hadoop-mapreduce-client-core-2.4.0.2.1.7.0-784.jar
hive-common-0.13.0.2.1.7.0-784.jar
hive-exec-0.13.0.2.1.7.0-784.jar
hive-jdbc-0.13.0.2.1.7.0-784.jar
hive-metastore-1.2.1.2.3.0.0-2557.jar
hive-service-0.13.0.2.1.7.0-784.jar
httpclient-4.2.5.jar
httpcore-4.2.5.jar
libthrift-0.9.0.jar
log4j-1.2.17.jar
slf4j-api-1.7.5.jar

For Hortonworks Data Platform versions prior to 2.3:

commons-codec-1.4.jar
commons-collections-3.2.1.jar
commons-configuration-1.6.jar
commons-logging-1.1.3.jar
hadoop-auth-2.4.0.2.1.7.0-784.jar
hadoop-common-2.4.0.2.1.7.0-784.jar
hadoop-mapreduce-client-core-2.4.0.2.1.7.0-784.jar
hive-common-0.13.0.2.1.7.0-784.jar
hive-exec-0.13.0.2.1.7.0-784.jar
hive-jdbc-0.13.0.2.1.7.0-784.jar
hive-service-0.13.0.2.1.7.0-784.jar
httpclient-4.2.5.jar
httpcore-4.2.5.jar
libthrift-0.9.0.jar
log4j-1.2.17.jar
slf4j-api-1.7.5.jar

BigSQL ./libs/horton/bigsql db2jcc.jar

The steps to obtain the JDBC files are different for each service provider. The following sections describe some general steps for obtaining the JDBC files for each vendor. See the Troubleshooting missing JDBC drivers for general information about finding the JDBC files. Refer to the documentation for your Hadoop service provider software for specific details about locating and obtaining the JDBC driver client files.

Obtaining IBM BigInsights 3 JDBC drivers

If you use the IBM BigInsights 3 BigSQL and Hive2 services, you can obtain the JDBC driver files as follows:

  1. Log in to the BigInsights Web Console.
  2. Click Quick Links > Download client library and development software.
  3. Select the following software bundles:
    • Hive JDBC package
    • BigSql and BigSqlv1 client libraries
  4. Save the JDBC files to the directory shown in Table 1.

Obtaining IBM BigInsights 4 JDBC drivers

If you use the IBM BigInsights 4 BigSQL and Hive2 services, you can obtain the JDBC driver files as follows:

  1. Copy the required JAR files (shown in Table 1) from the Hadoop master node. The files are typically located in the following directories:
    • For Hive:
      • /usr/iop/4.0.0.0/hive/lib/
      • /usr/iop/4.0.0.0/hadoop/
      • /etc/hadoop/conf/
    • For BigSQL:
      • /usr/ibmpacks/bigsql/4.0/db2/java/
  2. You can copy all of the JAR files if you want, but you need only the files shown in Table 1.
  3. Save the JDBC files to the directory shown in Table 1.

Obtaining Cloudera JDBC drivers

Drivers are available on the Cloudera website or you can download the driver files from a Cloudera installation package.

  • If you choose to download drivers from the Cloudera web site:

    Note that there are separate packages for Hive and Impala. The Hive package must be copied to ./libs/cloudera/hive/ and the Impala package to ./libs/cloudera/impala. Each download packages has several JDBC driver zip files. Ensure that you use the JDBC4 (not JDBC41) zip file for the compatible drivers. For both, Impala and Hive, JDBC4 drivers are the only supported version. The other JDBC versions that are included in the package are not compatible with IBM Fluid Query.

    Note: If you download and use the drivers from the Cloudera web site and encounter problems, you can use the drivers from a Cloudera installation kit as a workaround. Note that you must remove the drivers from the web download package first before you replace them with the other drivers.
  • If you choose to obtain the JDBC driver files from a Cloudera installation, use the following steps:
    1. Copy the required JAR files (shown in Table 1) from the Hadoop master node. The files are typically in the following directories:
      • /usr/lib/hive/lib/
      • /usr/lib/hadoop/client/
      • /opt/cloudera/parcels/CDH/lib/hadoop/lib
    2. You can copy all of the JAR files if you want, but you need only the files shown in Table 1.
    3. Save the JDBC files to the directory shown in Table 1.
      Important:
      • For connections to the Impala service, the IBM Fluid Query features only support the Hive2 drivers.
      • Cloudera 5.4 uses Java 7 JDBC drivers. You can only use Java 7 drivers with IBM Fluid Query 1.6 and IBM Netezza Analytics 3.2.1. If you are using IBM Netezza Analytics version prior to 3.2.1, download and use the Java 6 drivers from a Cloudera 5.3 installation kit.

Obtaining Hortonworks JDBC drivers

If you use the Hortonworks Hive2 services, you could obtain the JDBC driver files as follows:

  1. Copy the required JAR files (shown in Table 1) from the Hadoop master node. The files are typically in the following two directories, where X.X.X.X-XXXX are the software version and build numbers for your service provider:
    • ./usr/hdp/X.X.X.X-XXXX/hive/lib/
    • ./usr/hdp/X.X.X.X-XXXX/hadoop/client/
    For example:
    • ./usr/hdp/2.2.0.0-2041/hive/lib/
    • ./usr/hdp/2.2.0.0-2041/hadoop/client/
  2. You can copy all of the JAR files if you want, but you need only the files shown in Table 1.
  3. Save the JDBC files to the directory shown in Table 1.
Important: The JDBC drivers for Hortonworks version 2.3 are Java 7 drivers. You can only use Java 7 drivers with IBM Fluid Query 1.6 and IBM Netezza Analytics 3.2.1. If you are using IBM Netezza Analytics version prior to 3.2.1, download and use the Java 6 drivers from a Hortonworks 2.2 installation kit.

Troubleshooting missing JDBC drivers

If you encounter any problems finding or obtaining the JDBC files listed in Table 1, or if you see ClassNotFoundException error messages when trying to configure connections, you may be missing a JDBC file required for that service connection. Sometimes the files may be in other locations on the system. If you cannot find a JDBC file, you can try to search for the file on your service provider's master node.

  1. Log in to the service provider's master node as the root user.
  2. Run a command similar to the following, to search for the missing file:
    find / -name "missing-jar-name*.jar"
    For example:
    find / -name "commons-configuration*.jar"
  3. The command displays a pathname for the file if it is present on the system. If you cannot find the required JDBC file and the fqConfigure.sh script continues to show errors for missing files, contact your service provider for assistance with obtaining the missing file.


Feedback