Apache Spark SQL connection

Apache Spark SQL is a distributed data processing engine that provides a unified interface for querying structured data using SQL, with support for both batch and streaming processing.

Create a connection to Apache Spark SQL

To create the connection asset, you need these connection details:

  • Hostname or IP address: The hostname or IP address of the database.

  • Port number: The port of the database.

  • Deployment type: The environment type you want to connect to. After choosing a deployment type, more connection details appear for you to fill in:

    IBM watsonx.data on Cloud
    • Engine ID: The IBM watsonx.data Presto engine ID.
    • CRN: The IBM watsonx.data cloud resource name.

Credentials

  • API key: The application programming interface (API) key that identifies the calling application or user.

Certificates

  • Port is SSL-enabled: Enable this option if the port you have chosen is configured to accept SSL connections.

If the Port is SSL-enabled option is enabled, fill in the following fields:

  • SSL certificate: Enter the SSL certificate of the host to be trusted
  • Hostname in SSL certificate: The hostname in SubjectAlternativeName or Common Name (CN) part of the SSL certificate
  • Validate the SSL certificate: Enable this option to validate that the SSL certificate returned by the host is trusted.

For Credentials and Certificates, you can use secrets if a vault is configured for the platform and the service supports vaults. For information, see Using secrets from vaults in connections.

Federal Information Processing Standards (FIPS) compliance

This connection cannot be created in a FIPS environment.

Apache Spark SQL setup

Learn more