Apache Spark SQL connection

Apache Spark SQL is a distributed data processing engine that provides a unified interface for querying structured data using SQL, with support for both batch and streaming processing.

Create a connection to Apache Spark SQL

To create the connection asset, you need these connection details:

  • Hostname or IP address: The hostname or IP address of the database.

  • Port number: The port of the database.

  • Deployment type: The environment type you want to connect to. After choosing a deployment type, more connection details appear for you to fill in:

    IBM watsonx.data on Cloud
    • Engine ID: The IBM watsonx.data Presto engine ID.
    • CRN: The IBM watsonx.data cloud resource name.

Credentials

  • API key: The application programming interface (API) key that identifies the calling application or user.

Certificates

  • Port is SSL-enabled: Enable this option if the port you have chosen is configured to accept SSL connections.

If the Port is SSL-enabled option is enabled, fill in the following fields:

  • SSL certificate: Enter the SSL certificate of the host to be trusted
  • Hostname in SSL certificate: The hostname in SubjectAlternativeName or Common Name (CN) part of the SSL certificate
  • Validate the SSL certificate: Enable this option to validate that the SSL certificate returned by the host is trusted.

Apache Spark SQL setup

Learn more