Install external libraries

Install external libraries to make them available to Data Collector stages.

You can install external libraries for the following stages:
  • Before you use the following stages, install JDBC drivers for the implementation that you want to use:
    • JDBC Multitable Consumer source
    • JDBC Query Consumer source
    • MySQL Binary Log source
    • Oracle Bulkload source
    • Oracle CDC source
    • Oracle CDC Client source
    • Oracle Multitable Consumer source
    • Oracle target
    • SAP HANA Query Consumer source
    • JDBC Lookup processor
    • JDBC Tee processor
    • SQL Parser processor, when using the database to resolve the schema
    • JDBC Producer target
    • JDBC Query executor

    For example, to use the JDBC Query Consumer source or the JDBC Producer target with Oracle, install the Oracle JDBC drivers.

  • You can install external Java libraries to call external Java code from the scripting processors: Groovy, Java, and Jython Evaluator.
  • You can call external Python modules from the Jython Evaluator processor.
  • You can install the DataStax Enterprise (DSE) Java driver to configure the Cassandra target to use DSE username and password authentication or Kerberos authentication.
  • Before you use the Google Bigtable target, install the BoringSSL library.
  • Before you use the JMS Consumer source or the JMS Producer target, install the JMS drivers for the implementation that you are using.

When installing an external library, you install it into the stage library that includes the stage. For example, to use an external Java library with the Groovy Evaluator processor, you install the Java library as an external library for the Groovy stage library, streamsets-datacollector-groovy_4_0-lib.

To use an external library with multiple stage libraries, install the external library into each stage library associated with the stages. For example, if you want to use a MySQL JDBC driver with the JDBC Lookup processor and with the MySQL Binary Log source, you install the driver as an external library for the JDBC stage library, streamsets-datacollector-jdbc-lib, and for the MySQL Binary Log stage library, streamsets-datacollector-mysql-binlog-lib.

To install an external library, add the external library as an external resource for the engine, as described in Setting up external resources.