Monitoring Azure Storage Service
The Azure Storage Service sensor is automatically deployed and installed after you install the Instana agent.
Supported information
Supported client-side tracing
For this technology, Instana supports client-side tracing for the following languages and runtimes:
Configuration
Azure Storage sensor can be disabled and it can be filtered by tags and resource groups. It is possible to configure Azure Storage sensor via agent configuration in <agentinstall_dir>/etc/instana/configuration.yaml by:
com.instana.plugin.azure.storage:
# Valid values: true, false
enabled: false # enabled (true) by default
# Comma separated list of tags in key:value format
include_tags:
# Comma separated list of tags in key:value format
exclude_tags:
# Comma separated list of resource groups
include_resource_groups:
# Comma separated list of resource groups
exclude_resource_groups:
Azure Storage sensor can be disabled. To disable monitoring of the Azure Storage services use the following configuration:
com.instana.plugin.azure.storage:
enabled: false
Multiple tags and resource groups can be defined, separated by a comma. Tags should be provided as a key-value pair separated by :. In order to make configuration easier, it is possible to define which tags and resource groups you want to include in discovery or exclude from discovery. In case of defining tag or resource group in both lists (include and exclude), exclude list has higher priority. If there is no need for services filtering, the configuration should not be defined. It's not mandatory to define all values in order to enable filtering.
To include services by tags into discovery use following configuration:
com.instana.plugin.azure.storage:
include_tags: # Comma separated list of tags in key:value format (e.g. env:prod,env:staging)
To exclude services by tags from discovery use following configuration:
com.instana.plugin.azure.storage:
exclude_tags: # Comma separated list of tags in key:value format (e.g. env:dev,env:test)
To include services by resource groups into discovery use following configuration:
com.instana.plugin.azure.storage:
include_resource_groups: # Comma separated list of resource groups (e.g. rg_prod,rg_staging)
To exclude services by resource groups from discovery use following configuration:
com.instana.plugin.azure.storage:
exclude_resource_groups: # Comma separated list of resource groups (e.g. rg_dev,rg_test)
Discovery filtering can be configured on the global level for all Azure services. In case of defining filters for Azure Storage service, global filters will be overridden. For more details about global Azure service discovery filtering visit Azure Configuration. Azure Storage has been recently extended with Azure Queues.
Metrics collection
To view the metrics, select Infrastructure in the sidebar of the Instana User interface, click a specific monitored host, and then you can see a host dashboard with all the collected metrics and monitored processes.
Metrics for Storage Service are pulled every minute, which is the resolution Azure provides for monitoring of these services.
Extension provides 3 new Metrics for Azure Queues that are being pulled every hour.
Configuration data
Storage
| Storage Service Details | Description |
|---|---|
| Name | The name of the Storage service instance. |
| Resource Group | The resource group of the Storage service instance. |
| Location | The region where the instance is located |
| Subscription Id | The subscription ID of the Storage service instance. |
| Type | Storage type |
| Kind | Indicates the type of storage account |
| State | Current deployment state of the instance. |
| Access Tier | The access tier of the storage account. |
| Blob Capabilities | Indicates if the storage account is upgraded with Azure Data Lake Gen2 capabilities (ADLS Gen2). |
Queues
| Queue Details | Description |
|---|---|
| Name | The name of the Queue. |
| Resource Group | The resource group of the Storage service instance where Queue belongs. |
| Location | The region where the instance is located. In this case that is a Storage. |
| Subscription Id | The subscription ID of the Storage service instance. |
| Type | Queue type |
Blobs
| Blob Details | Description |
|---|---|
| Name | The name of the Blob. |
| Resource Group | The resource group of the storage service instance where the blob belongs. |
| Location | The region where the instance is located. In this case, the region is storage. |
| Subscription Id | The subscription ID of the storage service instance. |
| Type | Blob type |
Performance metrics
| Storage Service Metric | Description |
|---|---|
| Transactions | |
| Total number of transactions | The number of requests made to a storage service or the specified API operation. This number includes successful and failed requests, as well as requests which produced errors. Use ResponseType dimension for the number of different type of response. |
| Ingress | |
| The total ingress in bytes | The amount of ingress data, in bytes. This number includes ingress from an external client into Azure Storage as well as ingress within Azure. |
| Average | The average ingress. |
| Minimum | The minimum ingress. |
| Maximum | The maximum ingress. |
| Egress | |
| The total egress in bytes | The amount of egress data, in bytes. This number includes egress from an external client into Azure Storage as well as egress within Azure. As a result, this number does not reflect billable egress. |
| Average | The average egress. |
| Minimum | The minimum egress. |
| Maximum | The maximum egress. |
| Server Latency | |
| Average | The average latency used by Azure Storage to process a successful request, in milliseconds. This value does not include the network latency specified in AverageE2ELatency. |
| Minimum | The minimum latency. |
| Maximum | The maximum latency. |
| E2E Latency | |
| Average | The average end-to-end latency of successful requests made to a storage service or the specified API operation, in milliseconds. This value includes the required processing time within Azure Storage to read the request, send the response, and receive acknowledgment of the response. |
| Minimum | The minimum latency. |
| Maximum | The maximum latency. |
| Availability | |
| Average | The percentage of availability for the storage service or the specified API operation. Availability is calculated by taking the TotalBillableRequests value and dividing it by the number of applicable requests, including those that produced unexpected errors. All unexpected errors result in reduced availability for the storage service or the specified API operation. |
| Minimum | The minimum availability. |
| Maximum | The maximum availability. |
| Queue Capacity | |
| Average | The Queue capacity for the storage service or the specified API operation represented as number. |
| Queue Count | |
| Average | The queue count for the storage service or the specified API operation that is represented as a number. |
| Queue Message Count | |
| Average | The queue message count for the storage service or the specified API operation that is represented as a number. |
| Blob Capacity | |
| Average | The amount of storage that is used by the blob service of the storage account in bytes. |
| Blob Count | |
| Average | The number of blob objects that are stored in the storage account. |
| Blob Container Count | |
| Average | The number of blob containers in the storage account. |
| Blob Ingress | |
| Total | The total amount of ingress data, in bytes. This number includes ingress from an external client into the Azure storage and ingress within Azure. |
| Average | The average amount of ingress data. |
| Minimum | The minimum amount of ingress data. |
| Maximum | The maximum amount of ingress data. |
| Blob Egress | |
| Total | The total amount of egress data. This number includes egress to an external client from Azure storage and egress within Azure. As a result, this number does not reflect billable egress. |
| Average | The average amount of egress data. |
| Minimum | The minimum amount of egress data. |
| Maximum | The maximum amount of egress data. |
| Blob Server Latency | |
| Average | The average time used to process a successful request by Azure storage. This value does not include the network latency that is specified in SuccessE2ELatency. |
| Minimum | The minimum server latency of blob metrics. |
| Maximum | The maximum server latency of blob metrics. |
| Blob E2E Latency | |
| Average | The average end-to-end (E2E) latency of successful requests made to a storage service or the specified API operation, in milliseconds. This value includes the required processing time within Azure Storage to read the request, send the response, and receive acknowledgment of the response. |
| Minimum | The minimum end-to-end latency of blob metrics. |
| Maximum | The maximum end-to-end latency of blob metrics. |
Troubleshooting
Azure API throttling (429 errors)
Issue: When you monitor many storage accounts with many blob containers and queues, you might encounter Azure API throttling errors (HTTP 429) in the agent logs.
Resource calculation: For sizing purposes, each blob container and each queue counts as an individual resource, not only the storage account itself. For more information, see the sizing table.
Solutions:
- Reduce the monitoring scope by using include or exclude filters to monitor only critical storage accounts:
com.instana.plugin.azure.storage: include_tags: env:prod,critical:true # or include_resource_groups: rg_production_storage # or exclude non-critical accounts exclude_tags: env:dev,env:test - Distribute resources across multiple Azure subscriptions: Azure throttling limits are applied per subscription. If you have many storage accounts, consider organizing them across multiple Azure subscriptions. Distributing resources can help reduce throttling issues.
- Disable the Azure Storage sensor if detailed storage monitoring is not required:
com.instana.plugin.azure.storage: enabled: false
For more information about Azure throttling limits, see Azure Resource Manager request limits and throttling.
For more information about Azure service sizing, see Agent sizing and resource scalability for multi-subscription monitoring.