Monitoring BigQuery tables

Integrate BigQuery with Databand to monitor your BigQuery projects, datasets, and tables. For more information, see Data warehouse monitoring.

Prerequisites

Before you can connect BigQuery with Databand you must have a Service account and a key. To create these:

  1. Go to the Service accounts page in your Google Cloud Platform console.

  2. Select the project that you want to monitor.

  3. Click Create service account.

  4. Enter the necessary information, then click Create and continue.

  5. Grant the following roles, then click Done.

    • BigQuery data viewer
    • BigQuery job user
    • BigQuery resource viewer

  6. From the list of service accounts, click the email address for the Service account you just created to open its configuration.

  7. Switch to the Keys tab.

  8. Click Add key > Create new key.

  9. Select JSON as the key type, then click Create.

  10. The new JSON key is saved to your computer and is ready for you to use in the configuration of your monitor in Databand.

Creating a BigQuery monitor

Follow these instructions to create a BigQuery monitor in the Databand UI:

  1. Go to the Integrations page, then click Add integration and select BigQuery as the integration type.
  2. Click Next.
  3. Provide an Integration name, and then click Browse file. When the file browser opens, locate and select the JSON key that you generated in step 9.
  4. Click Next to authenticate with BigQuery.

Choosing what to monitor

After you authenticate with BigQuery, select what you want Databand to monitor from the list of objects that your Service account has access to. You can select entire projects, entire datasets, or individual tables. After you make your selections, click Save to finalize the creation of your BigQuery monitor. You can now see your new BigQuery monitor in your list of Databand integrations.

If you choose to monitor an entire project or dataset, any future tables that are added to that project or dataset are automatically added to your monitored tables in Databand.

Known issues

  • Each BigQuery monitor in Databand is limited to 1,000 tables, but you can create as many monitors as you want.
  • The BigQuery monitor collects only QueryJob transactions. Any other transactions are not supported and are not displayed in the Databand UI.