Downloading watsonx.data system logs from Azure storage

IBM watsonx.data system logs are stored in Azure Blob storage and can be downloaded for analysis or to attach to support tickets.

Before you begin

Ensure that the following requirements are met:

  • Azure subscription access with appropriate permissions.
  • Azure CLI is installed and authenticated:
    az --version
  • Access to the storage account that contains watsonx.data system logs.
    Note: The watsonx.data system logs are stored in the Azure storage account that was configured for system logs as a postinstallation step, under the blob container: 
    wxd-system-logs
  • Additional requirements for automated download script usage:

    • Bash shell (version 4.0 or higher) for Linux, macOS, or WSL on Windows.

      Run this command in a terminal:

      bash --version
    • GNU date or BSD date (for date manipulation)

      • Linux: GNU date (pre-installed)

      • macOS: BSD date (preinstalled)

    • Script execute permissions

      Run this in a terminal:

      chmod +x ./download-wxd-system-logs.sh
  • Obtain your Instance ID:
    Your instance ID is a unique identifier for your watsonx.data deployment. Example instance ID: e123456a-cc12-1234-a1c2-cfd22abe17b1. To obtain your instance ID:
    1. Go to the home page of your watsonx.data instance.
    2. Hover over the left side panel, and select "Instance details" from the options at the bottom.

      Your instance ID is displayed in the instance details section.

  • Obtaining Your Cluster ID

    The cluster ID can be found as a tag within the Kubernetes cluster associated to your watsonx.data instance.

    To locate this:
    1. Navigate in the Azure portal to your resource group.
    2. Locate and select the Kubernetes service corresponding to the cluster associated with your watsonx.data instance.
    3. On the left-side panel select the Overview page.
    4. Locate the Tags section of your Kubernetes cluster, and find the tag named "wxd-resource-id".
    5. Copy the value associated with this tag, this will be your cluster ID.

      Example cluster ID: `c51b8d7d-6fb2-418b-9efc-6fbb362c0e6b`

About this task

This topic covers both manual download methods and an automated script for bulk log retrieval.

Logs are organized in Azure Blob Storage by using the following directory path structure:
wxd-system-logs/<instance-id>/<cluster-id>/system/<YYYY-MM-DD>/<HH>/<component-name>/<pod-name>/<YYYY-MM-DD-HH>_pod-name_container-name.verified.log
Path components:
  • <instance-id>: Your unique watsonx.data instance identifier
  • <cluster-id>: Your cluster identifier
  • <YYYY-MM-DD>: Date in ISO format (for example, 2025-12-01)
  • <HH>: Hour in 24-hour format with leading zero (for example, 08, 16, 23)
  • <pod-name>: Kubernetes pod name
  • <component-name>: Name of the watsonx.data component (for example, argus-fluentbit)
  • Log file name format: <YYYY-MM-DD-HH>_pod-name_container-name.verified.log
Example path:
wxd-system-logs/e123456a-cc12-1234-a1c2-cfd22abe17b1/c51b8d7d-6fb2-418b-9efc-6fbb362c0e6b/system/2025-12-01/16/argus-fluentbit/argus-fluentbit-p285c/2025-12-01-16_argus-fluentbit-p285c_fluent-bit.verified.log

Procedure

You can download logs by using an automated script or by using a manual method.

  1. To download logs by using an automated script, run the following command: 
    ./download-wxd-system-logs.sh [Parameters]
    Required parameters:
    • -i, --instance-id <id> - Your watsonx.data instance ID (required)
    • -c, --cluster-id <id> - Your cluster ID (required)
    • -s, --start-date <date> - Start date in YYYY-MM-DD format (required)
    • -e, --end-date <date> - End date in YYYY-MM-DD format (required)
    • -n, --storage-account-name <name> - Azure storage account name (required)
    Optional parameters:
    • -d, --destination <dir> - Destination directory (default: ./downloaded-logs in current working directory)
      Note: If a custom destination directory is specified, it is created automatically if it doesn't exist.
    • -k, --account-key <key> - Storage account key for authentication
    • -a, --auto-key - Automatically retrieve and use account key
    • -h, --help - Show help message
    Note: The automated script supports the following authentication methods:
    • Azure CLI Login (Default): Uses your Azure CLI authentication
    • Account Key: Uses a storage account key for authentication
    Example 1: Downloading logs by using Azure CLI login authentication method
    ./download-wxd-system-logs.sh \
  --instance-id e123456a-cc12-1234-a1c2-cfd22abe17b1 \
  --cluster-id c51b8d7d-6fb2-418b-9efc-6fbb362c0e6b \
  --storage-account-name your-storage-account-name \
  --start-date 2025-11-28 \
  --end-date 2025-12-01
    Example 2: Downloading logs by using auto-retrieved account key
    ./download-wxd-system-logs.sh \
  --instance-id e123456a-cc12-1234-a1c2-cfd22abe17b1 \
  --cluster-id c51b8d7d-6fb2-418b-9efc-6fbb362c0e6b \
  --storage-account-name your-storage-account-name \
  --start-date 2025-11-28 \
  --end-date 2025-12-01 \
  --auto-key
    Example 3: Downloading logs by specifying the custom destination directory
    ./download-wxd-system-logs.sh \
  -i e123456a-cc12-1234-a1c2-cfd22abe17b1 \
  -c c51b8d7d-6fb2-418b-9efc-6fbb362c0e6b \
  -n your-storage-account-name \
  -s 2025-11-28 \
  -e 2025-12-01 \
  -d /path/to/custom/directory
    Example 4: Downloading logs by specifying the account key
    ./download-wxd-system-logs.sh \
  --instance-id e123456a-cc12-1234-a1c2-cfd22abe17b1 \
  --cluster-id c51b8d7d-6fb2-418b-9efc-6fbb362c0e6b \
  --storage-account-name your-storage-account-name \
  --start-date 2025-11-28 \
  --end-date 2025-12-01 \
  --account-key "your-storage-account-key-here"
    The automated script output contains the following values:
    • Displays the configuration being used.
    • Shows progress as it downloads logs for each date.
    • Creates the destination directory if it doesn't exist.
    • Downloads all logs matching the specified date range.
    • Displays a completion message with the location of downloaded logs.
    Example script output:
    Configuration:
  Storage Account: your-storage-account-name
  Container: wxd-system-logs
  Instance ID: e123456a-cc12-1234-a1c2-cfd22abe17b1
  Cluster ID: c51b8d7d-6fb2-418b-9efc-6fbb362c0e6b
  Start Date: 2025-11-28
  End Date: 2025-12-01
  Destination: ./downloaded-logs
  Auth Method: Azure CLI Login

Downloading logs from 2025-11-28 to 2025-12-01...
Downloading logs for 2025-11-28...
Downloading logs for 2025-11-29...
Downloading logs for 2025-11-30...
Downloading logs for 2025-12-01...

Download complete! Logs saved to: ./downloaded-logs
  2. To download logs by using the manual method, complete the following steps:
    1. Authenticate with Azure 
      az login
az account set --subscription <subscription-id>
    2. Assign Required Permissions (Optional)

      This step is optional. If you prefer not to assign RBAC roles, you can use the account key authentication method instead (see the following examples).

      To use Azure CLI login authentication, you need the Storage Blob Data Reader role. If you don't have this role, request it from your Azure administrator or assign it yourself if you have sufficient permissions.

      Run the following command:
      # Get your user object ID
USER_OBJECT_ID=$(az ad signed-in-user show --query id -o tsv)

# Get the storage account resource ID
STORAGE_ACCOUNT_ID=$(az storage account show \
  --name $STORAGE_ACCOUNT_NAME \
  --query id -o tsv)

# Assign the Storage Blob Data Reader role
az role assignment create \
  --assignee $USER_OBJECT_ID \
  --role "Storage Blob Data Reader" \
  --scope $STORAGE_ACCOUNT_ID
      Note: Role assignments can take a few minutes to propagate.
    3. List Available Logs
      To see what logs are available for a specific date, run the following command:
      # Set your instance ID, cluster ID, and storage account name
INSTANCE_ID="e123456a-cc12-1234-a1c2-cfd22abe17b1"
CLUSTER_ID="c51b8d7d-6fb2-418b-9efc-6fbb362c0e6b"
STORAGE_ACCOUNT_NAME="your-storage-account-name"

# List logs for a specific date (using login auth)
az storage blob list \
  --container-name wxd-system-logs \
  --account-name $STORAGE_ACCOUNT_NAME \
  --prefix "$INSTANCE_ID/$CLUSTER_ID/system/2025-12-01/" \
  --auth-mode login \
  --output table
      To see what logs are available for a specific date by using account key authentication, run the following command:
      # First, retrieve the account key
ACCOUNT_KEY=$(az storage account keys list \
  --account-name $STORAGE_ACCOUNT_NAME \
  --query '[0].value' -o tsv)

# List logs for a specific date (using account key)
az storage blob list \
  --container-name wxd-system-logs \
  --account-name $STORAGE_ACCOUNT_NAME \
  --prefix "$INSTANCE_ID/$CLUSTER_ID/system/2025-12-01/" \
  --account-key "$ACCOUNT_KEY" \
  --output table
    4. Download Logs
      To download a single log file by using authentication mode login, run the following command: 
      az storage blob download \
  --container-name wxd-system-logs \
  --account-name $STORAGE_ACCOUNT_NAME \
  --name "$INSTANCE_ID/$CLUSTER_ID/system/2025-12-01/16/argus-fluentbit/argus-fluentbit-p285c/2025-12-01-16_argus-fluentbit-p285c_fluent-bit.verified.log" \
  --file "./downloaded-logs/2025-12-01-16_argus-fluentbit-p285c_fluent-bit.verified.log" \
  --auth-mode login
      To download a single log file by using account key, run the following command: 
      az storage blob download \
  --container-name wxd-system-logs \
  --account-name $STORAGE_ACCOUNT_NAME \
  --name "$INSTANCE_ID/$CLUSTER_ID/system/2025-12-01/16/argus-fluentbit/argus-fluentbit-p285c/2025-12-01-16_argus-fluentbit-p285c_fluent-bit.verified.log" \
  --file "./downloaded-logs/2025-12-01-16_argus-fluentbit-p285c_fluent-bit.verified.log" \
  --account-key "$ACCOUNT_KEY"
      To download all logs for a specific date by using authentication mode login, run the following command: 
      # Create destination directory
mkdir -p ./downloaded-logs

# Download all logs for a date
az storage blob download-batch \
  --account-name $STORAGE_ACCOUNT_NAME \
  --source wxd-system-logs \
  --destination ./downloaded-logs \
  --pattern "$INSTANCE_ID/$CLUSTER_ID/system/2025-12-01/*" \
  --auth-mode login
      To download all logs for a specific date by using account key, run the following command: 
      # Create destination directory
mkdir -p ./downloaded-logs

# Download all logs for a date
az storage blob download-batch \
  --account-name $STORAGE_ACCOUNT_NAME \
  --source wxd-system-logs \
  --destination ./downloaded-logs \
  --pattern "$INSTANCE_ID/$CLUSTER_ID/system/2025-12-01/*" \
  --account-key "$ACCOUNT_KEY"
      To download logs for a date range by using authentication mode login, run the following command: 
      # Download logs for multiple dates
for date in 2025-11-28 2025-11-29 2025-11-30 2025-12-01; do
  echo "Downloading logs for $date..."
  az storage blob download-batch \
    --account-name $STORAGE_ACCOUNT_NAME \
    --source wxd-system-logs \
    --destination ./downloaded-logs/$date \
    --pattern "$INSTANCE_ID/$CLUSTER_ID/system/$date/*" \
    --auth-mode login
done
      To download logs for a date range by using account key, run the following command: 
      # Download logs for multiple dates
for date in 2025-11-28 2025-11-29 2025-11-30 2025-12-01; do
  echo "Downloading logs for $date..."
  az storage blob download-batch \
    --account-name $STORAGE_ACCOUNT_NAME \
    --source wxd-system-logs \
    --destination ./downloaded-logs/$date \
    --pattern "$INSTANCE_ID/$CLUSTER_ID/system/$date/*" \
    --account-key "$ACCOUNT_KEY"
done