Monitoring AWS S3
The AWS S3 sensor is automatically deployed and installed after you install the Instana agent.
Note: Learn about the other supported AWS services with our AWS documentation.
- Sensor (Data Collection)
- Required Permissions
For Instana agent to be able to collect metrics from S3 instance, paid (request & data transfer) metrics need to be enabled. More on this topic.
Sensor (Data Collection)
- S3 Bucket Details
- Bucket Name
- Bucket Owner Id
- Bucket Owner Name
- Bucket Created At
- Grouping zone (region)
|All (Request)||The total number of HTTP requests made to an Amazon S3 bucket, regardless of type. If you’re using a metrics configuration with a filter, then this metric only returns the HTTP requests made to the objects in the bucket that meet the filter's requirements.|
|Get (Request)||The number of HTTP GET requests made for objects in an Amazon S3 bucket. This doesn't include list operations.|
|Put (Request)||The number of HTTP PUT requests made for objects in an Amazon S3 bucket.|
|Delete (Request)||The number of HTTP DELETE requests made for objects in an Amazon S3 bucket. This also includes Delete Multiple Objects requests. This metric shows the number of requests, not the number of objects deleted.|
|Head (Request)||The number of HTTP HEAD requests made to an Amazon S3 bucket.|
|Post (Request)||The number of HTTP POST requests made to an Amazon S3 bucket.|
|List (Request)||The number of HTTP requests that list the contents of a bucket.|
|Bytes Downloaded||The number bytes downloaded for requests made to an Amazon S3 bucket, where the response includes a body.|
|Bytes Uploaded||The number bytes uploaded that contain a request body, made to an Amazon S3 bucket.|
|4xx Errors||The number of HTTP 4xx client error status code requests made to an Amazon S3 bucket with a value of either 0 or 1. The average statistic shows the error rate, and the sum statistic shows the count of that type of error, during each period.|
|5xx Errors||The number of HTTP 5xx server error status code requests made to an Amazon S3 bucket with a value of either 0 or 1. The average statistic shows the error rate, and the sum statistic shows the count of that type of error, during each period.|
|First Byte Latency||The per-request time from the complete request being received by an Amazon S3 bucket to when the response starts to be returned.|
|Total Request Latency||The elapsed per-request time from the first byte received to the last byte sent to an Amazon S3 bucket. This includes the time taken to receive the request body and send the response body, which is not included in FirstByteLatency.|
In order to have an indicator for public buckets, the following permission needs to be added:
A bucket policy for each specific S3 bucket should also be defined. For more information on how to add the bucket policy, see the AWS documentation.
Metrics for S3 are pulled every 60 seconds, this can be changed via agent configuration in
com.instana.plugin.aws.s3: cloudwatch_period: 300
To disable monitoring of S3 instances use the following configuration:
com.instana.plugin.aws.s3: enabled: false
To configure the specific AWS Sensor to use proxy configuration, add the following agent configuration settings:
com.instana.plugin.aws.s3: proxy_host: 'example.com' # proxy host name or ip address proxy_port: 3128 # proxy port proxy_protocol: 'HTTP' # proxy protocol: HTTP or HTTPS proxy_username: 'username' # OPTIONAL: proxy username proxy_password: 'password' # OPTIONAL: proxy password
Monitoring multiple AWS accounts
Refer to the Monitoring multiple AWS accounts documentation to set up monitoring of multiple AWS accounts with one AWS agent in the same region.
AWS named profiles approach
To override which profiles should be used to monitor S3, use the following configuration:
com.instana.plugin.aws.s3: profile_names: - 'profile2' - 'profile3'
Note: Defining profiles on service level will override the global AWS profiles configuration.
AWS STS approach
To override which IAM Roles should be used to monitor S3, use the following configuration:
com.instana.plugin.aws.s3: role_arns: - 'arn:aws:iam::<account_1_id>:role/<role_1_name>' - 'arn:aws:iam::<account_2_id>:role/<role_2_name>'
Note: Defining IAM roles on service level will override the global AWS IAM roles configuration.
Multiple tags can be defined, separated by a comma. Tags should be provided as a key-value pair separated by :. In order to make configuration easier, it is possible to define which tags you want to include in discovery or exclude from discovery. In case of defining tag in both lists (include and exclude), exclude list has higher priority. If there is no need for services filtering, the configuration should not be defined. It’s not mandatory to define all values in order to enable filtering.
Users are able to specify how often sensors will poll the AWS tagged resources using the
tagged-services-poll-rate configuration property (default 300 seconds).
Note: Tags are only available in conjunction with the AWS Agent.
To define how often sensors will poll the tagged resources use following configuration:
com.instana.plugin.aws: tagged-services-poll-rate: 60 #default 300
To include services by tags into discovery use following configuration:
com.instana.plugin.aws.s3: include_tags: # Comma separated list of tags in key:value format (e.g. env:prod,env:staging)
To exclude services by tags from discovery use following configuration:
com.instana.plugin.aws.s3: exclude_tags: # Comma separated list of tags in key:value format (e.g. env:dev,env:test)
AWS services without tags will be monitored by default but can be excluded by setting the
include_untagged field to
com.instana.plugin.aws.s3: include_untagged: false # True value by default