Cloudability Connect Custom Data (FOCUS Ingress)

Edit online

The  FinOps Open Cost and Usage Specification ( FOCUS)  is a community-driven effort to develop a standard schema for cloud, SaaS, and other billing data. The primary goal of the FOCUS specification is to make it easier to understand, report on, and manage cloud costs. The FOCUS specification is intended to be adaptable across a variety of cloud service providers and SaaS product sources and defines columns (dimensions and metrics), column-specific requirements, and attributes (spec-wide requirements).

You can bring any cost and usage data from additional data sources as long as it has been formatted into the FOCUS schema and adheres to the columns that are required, as defined by the specification. The FOCUS Specifications can be found here with specific columns additionally here . Please note that we currently support FOCUS version 1.0.
Note:

Custom Data credentialing option should not be used for CSPs that we already support natively (e.g., AWS, Azure, etc.), as some features, such as rightsizing, will not be supported.

Steps for integration

Data Source Management

All the data that is uploaded to any of the storage services must be the entire month-to-date data.

Also, the data must follow a specific structure as:

<Root Directory>/<Optional_Sub_Directories>/<Vendor>/<Report_Period>/<Epoch Time Folder>/<prefix>-Manifest.json

The M in Manifest should always be capitalized.

  • Root Directory : A prefix like AWS bucket name, Azure container name or GCS bucket name

  • Optional Sub Directories : An optional field for customer. They may or may not have additional sub directories

  • Vendor : A vendor name which would help in identifying the vendor type

  • Report Period : Report Period for which the data is available. This must be in the formats YYYYMMDD-YYYYMM(+1)DD . For example, for report period 2023-02 the name would look like →20230201-20230301.

  • Epoch Time Folder : This will be used to identify the latest drop for month-to-date data.

  • Manifest : Metadata json file containing details of specific report period dataset.

All custom data sources via this capability needs to adhere to the FOCUS specifications.

Sample Manifest File 
{
		"compression": "GZIP",
		"content_type": "Parquet",
		"report_id": "uuid-f431e214843b3c19756368",
		"root_dir": "byod-input-prod",
		"all_report_keys": [
		"<complete/path/excluding/root_dir>/cost-and-usage-report-00001.snappy.parquet",
		"<complete/path/excluding/root_dir>/cost-and-usage-report-00002.snappy.parquet"
		],
		"updated_at": "2024-04-04T05:00:22Z",
		"focus_version": "1.0"
	}

File Content Type : JSON

Header / Field Header / Field definition Value : Required / Optional
compression

CSV : gzip

Parquet : All that are supported by Java Hadoop frameworks.

Optional
content type Parquet or CSV Required
report ID Randomly generated unique report identifier. Optional
root_dir S3 Bucket or Azure Directory or GCS bucket to fetch the report from. This will be dependent of the type of credentials chosen. Required
all report keys It is a nested JSON, that is the list of only new absolute file paths excluding the root directory. Required
updated at Last updated time of the report. Optional
focus version FOCUS version as per https://finops.dev/focus-spec The version must match the exact FOCUS Spec version. Required
Integration Steps
Note:

Once you have completed the credentialing process in Cloudability (Described below), you will see a green check mark next to your credential details, but Cloudability will still need to validate that the billing file and the manifest file comply with the requested formats. If you do not see any data in Cloudability after 24 hours, please reach out to the Apptio Support team.

We support AWS Simple Storage Service , Google Cloud Storage and Azure Blob Storage as the storage services for housing FOCUS-aligned data.

To add credentials for custom data in Cloudability , follow the steps below:

  1. In Cloudability , navigate to Settings > Vendor Credentials > Custome Data .

  2. Select Add a Credential. The Add a Credential panel opens.

  3. Click on Preferred Storage Provider .

AWS Simple Storage Service
  1. Enter Vendor for which the data needs to be ingested.
  2. Enter AWS Account ID .
  3. Enter AWS S3 bucket name under Root Directory .
  4. Enter optional sub prefixes under Sub Directories .
  5. Enter Manifest Prefix under Manifest Prefix .
  6. Click Generate template .

For more information, refer to Connect Amazon Web Services.

Azure Blob Storage
  1. Enter the Vendor for which the data needs to be ingested.
  2. Enter Azure Account ID.
  3. Enter Tenant ID .
  4. Enter Subscription ID .
  5. Enter the Resource Group name.
  6. Enter the Storage Account name.
  7. Enter Blob Container Name under Root Directory .
  8. Enter optional sub directories under Sub Directories .
  9. Enter Manifest Prefix under Manifest Prefix .
  10. Click Generate template .

For more information, refer to Connect Microsoft Azure .

Google Cloud Storage Simple Storage Service
  1. Enter the Vendor for which the data needs to be ingested.
  2. Enter GCP Account ID .
  3. Enter GCP GCS bucket name under Root Directory .
  4. Enter optional sub prefixes under Sub Directories .
  5. Enter Manifest Prefix under Manifest Prefix .
  6. Click Generate template .

For more information, refer to Connect Google Cloud.