Configuring JDBC access to Spark data sources

To configure the federated server to access Spark data sources, you must provide the federated server with information about the data sources and objects that you want to access. Spark that are accessed through the JDBC API are referred to in this text as Spark data sources.

Before you begin

The spark JDBC driver must be installed and configured on the computer that acts as the federated server.

Spark provides an JDBC driver for the AIX®, Linux operating systems.

For prerequisites and limitations that apply to all JDBC data sources, see .

About this task

Restrictions

  • IUD and Transparent IUD are not supported.
  • Method 1 is not supported for NNSTAT.
  • Spark type: Binary supports binary(256) for query and will be truncated when the lengh > 256.
  • Spark type: Binay and String will be truncated when the lengh > 32767 in passthru-mode session.
  • TINYINT, SMALLINT, INT, BIGINT, DECIMAL types return "-" when remote data overflow.
  • COMMIT and ROLLBACK is not supported.

Procedure

  1. Set up Spark and start thriftserver using $spark_path/sbin/start-thriftserver.sh.
  2. Find JDBC driver:hive-jdbc-0.14.0.2.2.4.2-2-standalone.jar and hadoop-common-2.6.0.2.2.4.2-2.jar, put them into the computer that acts as the federated server.
  3. Set CLASSPATH for jdbc driver and then JDBC wrapper can load JDBC driver to connect to spark. 
    export CLASSPATH=/$dirver_location/hive-jdbc-0.14.0.2.2.4.2-2-standalone.jar:/$dirver_location/hadoop-common-2.6.0.2.2.4.2-2.jar:$CLASSPATH