Setting up an AWS data export

This topic describes the data export that the AWS Billing target in Turbonomic will use to monitor billing data.

The AWS Billing target grants Turbonomic access to billing data from a data export stored in an S3 bucket. Turbonomic uses this data to optimize workloads with full cost awareness, discover discounts and billing family relationships, and visualize historical cloud expenses.

Important:

Set up the data export exactly as described in this topic to comply with Turbonomic requirements. Any additional configuration may not work with the AWS Billing target that is associated with the data export.

Level of support for data exports

Turbonomic supports only the following data exports:

(Recommended) Standard data export (CUR 2.0)
Legacy CUR export

All other data exports available in AWS are not supported.

Turbonomic supports a data export created at the management account, but not member accounts.

Data export delivery

AWS publishes the daily data export twice a day, but it may take up to 24 hours for AWS to deliver the data export to the S3 bucket. Charts in Turbonomic, such as the Workload Cost Breakdown chart, will start to show data 1 to 2 hours after data export delivery.

S3 bucket for the data export

The IAM user or role that you set up for Turbonomic requires access to the S3 bucket that contains your data export.

The AmazonS3FullAccess and Billing permissions are required to set up the bucket. For detailed setup instructions, see the AWS documentation.

Be sure to verify the bucket policy that AWS applies and that no service control policy blocks access to the bucket. To verify access, open the AWS CLI and run the following command:

aws s3api get-bucket-acl --bucket {name}

Replace {name} with the bucket name.

The result should be similar to the following example:

    "Owner": {
        "DisplayName": "XXX",
        "ID": "X"
    },
    "Grants": [
        {
            "Grantee": {
                "DisplayName": "XXX",
                "ID": "XXX",
                "Type": "XXX"
            },
            "Permission": "READ"
    ]

If access is blocked, you may see the following error:

when calling the GetBucketAcl operation: User: xxx is not authorized to perform: s3:GetBucketAcl on resource: "xxx" because no identity-based policy allows the s3:GetBucketAcl action

To resolve the error:

Update any service control policy that is blocking access to the S3 bucket. The following example shows a service control policy that unblocks access.

{
            "Effect": "Allow",
            "Action": [
                "s3:getObject",
                "s3:getBucketAcl"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "aws:RequestedRegion": "{region}"
                }
            }
        },

Replace {region} with the region that hosts the bucket.

Run the following command to verify that access is now unblocked.
```
aws s3api get-bucket-acl --bucket {name}
```

Setting up a standard data export (CUR 2.0)

Sign in to the Billing and Cost Management console.

https://console.aws.amazon.com/billing
In the page banner, find the dropdown menu of regions and then select the region for the S3 bucket.
In the navigation pane, find the Cost Analysis section and choose Data Exports.
Choose Create.

Choose Standard data export and configure the following settings.

Export name

Setting Instructions
Export name Specify your preferred name.

Setting	Instructions
Export name	Specify your preferred name.

Data table content settings

Setting	Instructions
Additional export content	Select Include resource IDs.
Time granularity	Choose Daily.

Column selection

By default, all columns are selected. To include only the columns that Turbonomic currently requires, select the following items:
- bill_payer_account_id
- identity_line_item_id
- line_item_currency_code
- line_item_line_item_type
- line_item_net_unblended_cost
- line_item_normalization_factor
- line_item_normalized_usage_amount
- line_item_operation
- line_item_product_code
- line_item_resource_id
- line_item_unblended_cost
- line_item_usage_account_id
- line_item_usage_amount
- line_item_usage_end_date
- line_item_usage_start_date
- line_item_usage_type
- pricing_public_on_demand_cost
- pricing_term
- pricing_unit
- product
- product_instance_type
- product_product_family
- product_sku
- reservation_reservation_a_r_n
- resource_tags
- savings_plan_savings_plan_a_r_n
- savings_plan_savings_plan_effective_cost

Data export delivery options

Setting	Instructions
Compression type and file format	Choose gzip - text/csv.
File versioning	Choose Create new data export file or Overwrite existing data export file.

Data export storage settings

Setting	Instructions
S3 bucket	Click Configure and then select an existing bucket or create a new one. If you created a new one, the region that you selected in a previous step is automatically specified.
S3 path prefix	Specify your preferred prefix, such as `daily`.

Review your settings and record the following information. You will need this information later when you add the AWS Billing target in the Turbonomic user interface.
- S3 bucket name, such as turbodataexport
- S3 path prefix, such as daily
- S3 bucket region, such as us-east-1
Click Create.

AWS can now deliver the data export to the S3 bucket. It could take up to 24 hours for AWS to deliver the first data export.

Setting up a Legacy CUR export

Sign in to the Billing and Cost Management console.

https://console.aws.amazon.com/billing
In the page banner, find the dropdown menu of regions and then select the region for the S3 bucket.
In the navigation pane, find the Cost Analysis section and choose Data Exports.
Choose Create.

Choose Legacy CUR export and configure the following settings.

Export name

Setting Instructions
Export name Specify your preferred name.

Setting	Instructions
Export name	Specify your preferred name.

Export content

Setting	Instructions
Additional export content	Select Include resource IDs.

Data export delivery options

Setting	Instructions
Report data time granularity	Choose Daily.
Report versioning	Choose Create new report version or Overwrite existing report.

Data export storage settings

Setting	Instructions
S3 bucket	Click Configure and then select an existing bucket or create a new one. If you created a new one, the region that you selected in a previous step is automatically specified.
S3 path prefix	Specify your preferred prefix, such as `daily`.

Review your settings and record the following information. You will need this information later when you add the AWS Billing target in the Turbonomic user interface.
- S3 bucket name, such as turbodataexport
- S3 path prefix, such as daily
- S3 bucket region, such as us-east-1
Click Create report.

AWS can now deliver the data export to the S3 bucket. It could take up to 24 hours for AWS to deliver the first data export.

Next step

In the Turbonomic user interface, add an AWS Billing target using the data export settings and the IAM user or role that you set up. For details, see this topic.