Connecting to Azure Event Hubs

To stream data to IBM Guardium® Insights, you must first connect to your data sources. Learn how to connect to Azure.

Before you begin

Notes:
  • Cloud database service protection for Azure event hubs works with Microsoft Azure SQL Database, Azure SQL Managed Instance, and Microsoft Azure Cosmos DB.
  • To use cloud database service protection with Azure, you need to be able to create Consumer Groups, which requires the Azure Standard Pricing Tier. You cannot use the $Default Consumer Group.
Important: If you are using Azure SQL Managed Instance, you must enable logging at both the server and database levels before you define the cloud database service account.
  1. Make sure that the ssh, 5671, and 5672 ports are open.
  2. Use SQL Server Management Studio (SSMS) to set up a and enable a Server Audit Specification that sends SQL Security Audit Events to the Event Hub.
  3. From your managed database, set up and enable the Database Audit Specification.
  4. On the Azure Managed Instance page, browse to Monitoring >Diagnostic Settings > Logs > Categories, and then select SQL Security Audit Event.

Gathering Microsoft Azure information

Before you can define a Guardium cloud database service account for Azure, you need to set up your Azure account or gather information about your existing account.

To use database activity monitoring, you need the following information about each Microsoft Azure event hub that you want to monitor. These parameters are created when you configure an Azure event hub. For detailed information about configuring Azure, see the Microsoft Azure documentation. Find or create the following Azure parameters:
  • Namespace: The Event Hubs namespace.
  • Event Hub Name: Created from within the Event Hubs namespace.
  • Shared access policy name and key: From the Event Hubs Namespace. To create a shared access policy, select Event Hubs Name > Shared Access Policy > your policy name to generate a shared access key.
    Note: Do not use the shared access policy in the Event Hub.

    Select Manage, Send, and Listen options for the Policy Name.

  • Consumer Group Name: From the Event Hubs Instance page for the selected Event Hub. From Entities, select or create a Consumer Group.
    Notes:

    If you use the same Consumer Group for multiple collectors, traffic is split between the collectors. If you create a Consumer Group for each collector, each collector gets its own copy of the traffic.

    You cannot use the $Default Consumer Group.

  • Storage Connection String: Create a storage account (from the Azure dashboard > All services) and then from Storage accounts, select Shared access signature to generate a shared access signature and connection string. The Storage account contains checkpoints for consumer progress in the Event Hubs partition. For example:
    BlobEndpoint=https://mystoragename.blob.core.windows.net/;QueueEndpoint=https://mystoragename.queue.core.windows.net/;FileEndpoint=https://mystoragename.file.core.windows.net/;TableEndpoint=https://mystoragename.table.core.windows.net/;SharedAccessSignature=sv=2019-12-12&ss=bfqt&srt=sco&sp=rwdlacupx&se=2025-09-16T00:54:06Z&st=2020-09-15T16:54:06Z&spr=https&sig=q%2FTuyiJqkNgfgdfgdfgdfgzaNj3V7Y0cr2EbLqol6Hg%3D
    Note: On the Shared access signature pane, change the expiry end date to meet your requirements.
  • Cluster resource Id: To find the cluster (or database) resource string:
    • For AzureSQL - From the Azure dashboard, browse to Properties > Resource ID.
    • For any Cosmos data source - The resource ID is the part of the URL that starts with /subscriptions and ends with the data source name. You can copy the resource ID from the URL, for example, if the URL for Microsoft Azure is:
      https://portal.azure.com/#@company.onmicrosoft.com/resource/subscriptions/8333367e-1234-467d-b3fc-5b78c5721df0/resourceGroups/rg1/providers/Microsoft.DocumentDb/databaseAccounts/ibmcosmostable1/overview

      Then the Cosmos resource ID is:

      /subscriptions/8333367e-1234-467d-b3fc-5b78c5721df0/resourceGroups/rg1/providers/Microsoft.DocumentDb/databaseAccounts/ibmcosmostable1

After you create your account and have the necessary information, you can define the cloud DB service accounts that you need.

Tips and Tricks

  • Before you start, create standard naming conventions to prevent later confusion. Consider including the name of the Event Hub and the name of the database that you are monitoring for each related element. For example, if the database name is use1-db5, use the following naming conventions:
    • Namespace: use1-ehn1
    • Shared Access: use1-ehn1-sa1
    • Event Hub: use1-db5-ehn1-eh3
    • Consumer Group: use1-db5-ehn1-eh3-cg
  • From the Guardium collector, make sure that outbound ports 443, 5671, and 5672 are available for the connections between the collector and Azure Event Hub.
  • When you create a namespace, consider selecting Enable Auto-Inflate.
  • Cosmos databases do not use usernames. Therefore, usernames are never returned from Cosmos.

If you are connecting to data sources for the first time, Guardium Insights guides you through your first connections as part of the getting started experience. To add more data sources (or work with data sources that are already defined), click Connections in the main menu. Open this menu by clicking the main menu icon (main menu))

Procedure

  1. Open the main menu at the upper left of the page (open this menu by clicking the main menu icon (main menu)) and then click Connections - or click View all connections in the Connections to Guardium Insights card on the Dashboards page. Then click View all or + in the Azure card.
  2. In the Azure Event Hubs page, click Add event hub.
    The Azure credentials page opens.
    Note: For more information about Azure credentials, see Cloud database service protection Azure setup.
  3. Set the Azure credentials:
    1. To set up a new connection, ensure that the Connect an account tab is active and complete these fields:
      • Create a name for your account: A unique name (with a minimum of 4 characters) used to identify your account in the future.
      • Add your access information: Enter your Azure Shared access policy name and Shared access policy key.
    2. To reuse an existing connection, click Use previous account and then select the account that you want to use.
  4. Click Next.
  5. Complete all the settings in the Add event hub info page:
    • Event Hub name: Enter the name of the event hub to connect to.
    • Namespace: Enter the namespace.
    • Database type: Select the type of database to which you are connecting.
    • Port: Specify the database port number.
    • Cluster resource id: Specify the cluster resource ID.
    • Consumer group name: Specify the Consumer group name.
    • Database DNS endpoint: Specify the database DNS endpoint (host).
    • Storage connection string: Specify the storage connection string.
  6. Click Next.
  7. Optional: Add database credentials: The credentials are necessary for setting up user blocking. Do not modify the default values.
    The database credentials that you can add (or view) are:
    • Database name: The database name.
    • Database host: The hostname - Do not change this value.
    • Username and Password: The username and password for this database.
  8. Click Connect and finish to create the connection.

What to do next

After you add a data source, it is scanned almost immediately. You manage your connections and connection credentials from the Connections page.

  • To delete a connection, click the connection checkbox and then click Remove in the banner that opens. You can select multiple connections to remove.
  • To edit a connection, select its Connection name link in the table. A window opens from which you can Enable or Disable the connection. In addition, you can see the status of the connection or click Edit icon to change the configuration for that connection. When you are done, click Save to save your changes and rescan the connection.
  • To download a CSV list of the connections in the table, click Download CSV icon. A list of the connections currently in the table is exported - it does not include any that are filtered out.
  • To refresh the list of connections, click Refresh.
  • You can filter connections by opening the Filter window (select the filter criteria and then click Apply filters).
  • To customize the columns in the table, click Customize columns. Then, under Customize columns, select the columns that you want to display in the table - and drag the columns to reorder them. Click Done when you finish.

From the list of Amazon Web Services and Azure connections, click the account entry in the Account column to open a window from which you can modify the account settings or delete the account. If you delete the account, all streams that were added for the account are also deleted.