How to use IBM App Connect with Google Cloud BigQuery

Enter the pagination type. Google Cloud BigQuery is an analytical data warehousing solution that enables organisations to analyse data across clouds.

Availability:
  • A connector in IBM App Connect on IBM CloudCloud-managed connector
  • A local connector in a Designer instance of IBM App Connect in containers (Continuous Delivery release)Local connector in containers (Continuous Delivery release) 11.0.0.10-r1 or later
  • A local connector in a Designer instance of IBM App Connect in containers (Extended Update Support)Local connector in containers (Extended User Support release)
  • A local connector in a Designer instance of IBM App Connect in containers (Long Term Support)Local connector in containers (Long Term Support release)

Connecting to Google Cloud BigQuery

Complete the connection fields that you see in the App Connect Designer Catalog page or flow editor. If necessary, work with your Google Cloud BigQuery administrator to obtain these values.

Google Cloud BigQuery connection fields:

Project ID
Your Google Cloud BigQuery project ID
  • Required: True
Dataset ID
The ID (schema name) of the dataset containing the tables and views that you want to work with in your Google Cloud BigQuery project. To work with tables or views from another dataset, you'll need to create a different account using that specific dataset ID.
  • Required: True
Client email
Also known as the service account ID. You can find or create this value in the 'APIs & Services' or 'IAM & Admin' section of the Google Cloud Console. For example, abc-222@bigqueryproject-22xxx.iam.gserviceaccount.com.
  • Required: True
Private key
A key used to establish the identity of the service account. Find or create this value in the Service Accounts page of the Google Cloud Console. For more information, see https://ibm.biz/googlebigquery_privatekey.
  • Required: True
To obtain these values for Google Cloud BigQuery, complete the following steps.
  1. Log in to the Google Cloud Console.
  2. On the dashboard, select an existing project, or click Create project.
  3. Enter a project name and organization, then click Create.

    When the project is created, the Project ID is shown on the dashboard.

  4. On the Getting Started tile, click Explore and enable APIs, then click Enable APIs and services.
  5. Search for and select BiqQuery API, then click Manage.
  6. Click Credentials, then click Credentials in APIs & Services.
  7. Click Create credentials, then Service account, and follow the on-screen instructions to create a service account.
  8. When the service appears on the Credentials page, click the service account, then click Add key, then Create new key.
  9. Select JSON as the key type, then click Create.
  10. Save the JSON file. This file contains the Client ID and Private key.

General considerations

Before you use App Connect Designer with Google Cloud BigQuery, take note of the following considerations:

  • (General consideration) You can see lists of the trigger events and actions that are available on the Catalog page of the App Connect Designer.

    For some applications, the events and actions in the catalog depend on the environment (IBM® Cloud Pak for Integration or App Connect on IBM Cloud) and whether the connector supports configurable events and dynamic discovery of actions. If the application supports configurable events, you see a Show more configurable events link under the events list. If the application supports dynamic discovery of actions, you see a Show more link under the actions list.

  • (General consideration) If you are using multiple accounts for an application, the set of fields that is displayed when you select an action for that application can vary for different accounts. In the flow editor, some applications always provide a curated set of static fields for an action. Other applications use dynamic discovery to retrieve the set of fields that are configured on the instance that you are connected to. For example, if you have two accounts for two instances of an application, the first account might use settings that are ready for immediate use. However, the second account might be configured with extra custom fields.

Events and actions

Google Cloud BigQuery events

These events are for changes in this application that trigger a flow to start performing the actions in the flow.

Note: Events are not available for changes in this application. You can trigger a flow in other ways, such as at a scheduled interval or at specific dates and times.

Google Cloud BigQuery actions

These are actions on this application that you want a flow to complete.

Datasets
Create dataset
Retrieve datasets
Update dataset
Delete dataset
Jobs
Retrieve jobs
Tables
Retrieve tables
Update table
Delete table
Restore deleted table
Copy table
Export data
Load data
Views
Retrieve views
Delete view