Configuring Google BigQuery for External S-TAPs

Include External S-TAPs in your Google BigQuery configuration for Guardium®.

You can use External S-TAPs to intercept Google BigQuery traffic in three different ways:
  • The BigQuery client libraries
  • BigQuery JDBC driver from CData Software
  • Google Cloud Console

You can deploy External S-TAP for Google BigQuery either with Kubernetes as described in Deploying External S-TAP from the Guardium UI or with Docker, as described in Deploying External S-TAP manually.

If you use Kubernetes, set the following Kubernetes environment variables for External S-TAP:
  • STAP_CONFIG_PROXY_DB_HOST=bigquery.googleapis.com
  • STAP_CONFIG_DB_0_REAL_DB_PORT=443
  • STAP_CONFIG_DB_0_DB_TYPE=bigquery
If you use Docker, set the following variables in the deployment script:
  • db_host: bigquery.googleapis.com
  • db_type: bigquery
  • db_port: 443

How you configure Google BigQuery depends on which method you want to use to intercept traffic. Follow the directions for the method you want to use.

Configure the certificate for BigQuery client libraries or CData BigQuery JDBC driver

For either the BigQuery client libraries or the CData BigQuery JDBC driver, you need to take the following steps:

  1. Generate a replacement certificate and private key for External S-TAP that meet the following requirements:
    • The key type is RSA.
    • Common Name (CN) - bigquery.googleapis.com
    • Subject Alternative Names (SAN) - DNS:*.cloud.google.com, DNS:clients6.google.com, DNS:*.clients6.google.com, DNS:bigquery.googleapis.com
    • The certificate is signed by a CA key pair.
  2. Deploy the certificate and private key. Combine the certificate and private key into a single file and put them into a Docker volume or Kubernetes secret.
    • If you use a Kubernetes cluster, set the External S-TAPSTAP_CONFIG_PROXY_PEM_PATH environment variable to the path of the file that contains the combined certificate and key inside the container.
    • If you use Docker on a virtual machine, store the certificate and private key by using the “store certificate external_stap_signing” cli command which creates a certificate token for the External S-TAP. For more information, see Certificate CLI Commands.
  3. Configure DNS so that the clients you want to monitor get the IP address of the External S-TAP for the following domain names:
    • bigquery.googleapis.com
    • console.cloud.google.com
    • cloudconsole-pa.clients6.google.com
    • clients6.google.com
    Note: To test on a local machine, you can use an entry into /etc/hosts. For production, Guardium suggests that you use a DNS system (such as BIND 9).

Configure the BigQuery API Client Libraries

To use the Google BigQuery API Client Libraries with External S-TAP, you first need to install the client libraries, as described in the Google Cloud BigQuery documentation. Then, configure the client library to trust the new External S-TAP certificate as well as the certificates that make up the signing chain.

To use the bq cli tool, you must append the External S-TAP certificate and signing chain to the cacerts.txt file, which you can find in one of the following locations (depending on your python version).

  • /google-cloud-sdk/platform/bq/third_party/httplib2/python3/cacerts.txt
  • /google-cloud-sdk/platform/bq/third_party/httplib2/python2/cacerts.txt

Configure the CData BigQuery JDBC driver

If you plan to use the CData BigQuery JDBC driver to connect through External S-TAP, you first need to install and configure the CData BigQuery JDBC driver. For more information, see the CData JDBC Driver for Google BigQuery documentation.

After you configure the CData BigQuery JDBC driver for your project, data set, and OAuth, set the following properties:

  • ProxyAuthScheme=NONE
  • ProxyPort=443
  • ProxySSLType=ALWAYS
  • ProxyServer= The IP address of your External S-TAP. If not set correctly, the Google BigQuery servers reject the connection with a "handshake failure."
  • SSLServerCert= The path to the file that contains the External S-TAP certificate and signing chain.

Configuring Google Cloud Console (BigQuery web UI)

To use an External S-TAP to intercept data from the Google Cloud Console, create or select a Google Cloud project.
Note: BigQuery is automatically enabled in Cloud Console projects.
  1. Generate a replacement certificate and private key for External S-TAP that meet the following requirements:
    • The key type is ECDSA with secp256r1 (prime256v1)
    • Common Name (CN) - bigquery.googleapis.com.
    • Subject Alternative Names (SAN) - DNS:*.cloud.google.com, DNS:clients6.google.com, DNS:*.clients6.google.com, DNS:bigquery.googleapis.com.
    • The certificate is signed by a CA key pair.
  2. Deploy the certificate and private key. Combine the certificate and private key into a single file and put them into a Docker volume or Kubernetes secret.
    • If you use a Kubernetes cluster, set the External S-TAPSTAP_CONFIG_PROXY_PEM_PATH environment variable to the path of the file that contains the combined certificate and key inside the container.
    • If you use Docker on a virtual machine, store the certificate and private key by using the store certificate external_stap_signing cli command which creates a certificate token for the External S-TAP. For more information, see Certificate CLI Commands.
  3. Configure DNS so that the clients you want to monitor get the IP address of the External S-TAP for the following domain names:
    • bigquery.googleapis.com
    • console.cloud.google.com
    • cloudconsole-pa.clients6.google.com
    • clients6.google.com
    Note: To test on a local machine, you can use an entry into /etc/hosts. For production, Guardium suggests that you use a DNS system (such as BIND 9).
  4. Configure the web browser. Make the following changes to the client web browser:
    Tip: This example uses the Firefox web browser. For other browsers, the parameters might be slightly different. From Firefox, change the settings from the preference page. Enter about:config in the Firefox address bar, and click through the warnings to open thepreference page.
    1. When available, the Google BigQuery Cloud Console uses HTTP 2. Since Sniffer does not support HTTP 2, you need to disable HTTP 2 support in the browser to force it to use HTTP 1.1.
      • network.http.spdy.enabled = false
      • network.http.spdy.enabled.http2 = false
    2. The client web browser needs to trust the External S-TAP certificate and signing chain.
      • security.osclientcerts.autoload = true
      • security.enterprise_roots.enabled = true
    3. Add the External S-TAP certificate (along with the other certificates in the signing chain) to your browser's trusted certificates.

      On Linux, copy the External S-TAP certificate and the rest of the certificates in its signing chain to the following location:

      /etc/pki/ca-trust/source/anchors/
      In Windows, take the following steps:
      • Break all of the certificates into separate certificate files with a .crt file extension.
      • Import the certificates into a Windows trusted CA by clicking each certificate file to display an import dialog.
      • If the certificate is the root CA, select Trusted root Certificate Authorities. For intermediate certificates, select Intermediate Certificate Authortities.
    4. After the certificates are in place, you need to update the certificate truststores. Run the following command as root: 
      update-ca-trust

      If your browser is open, you need to close and reopen it.