CSV Pricing

Learn how to set up and configure a pricing specification.

In the next section we’ll cover detailed instructions for building a pricing spec. For now, we’ll show an example and briefly explain each column.

Generally, Version, AssetClass, Unit and PricePerUnit are required for each row. Additionally, at least one mode of further specification is required – InstanceType or LabelName and LabelValue. In the section, below, Building a price spec, we share more details about which combinations of columns are valid for each asset class.

Once a CSV has been defined per the pricing spec. options, it must be provided to Kubecost via a ConfigMap. The ConfigMap can be pre-created or defined as part of your Helm values. To pre-create a ConfigMap in the kubecost namespace, see the below command.

kubectl create configmap -n kubecost kubecost-enterprise-pricing --from-file spec.csv

If your Kubecost install is not in namespace kubecost, make sure you alter the above command. We recommend using the default ConfigMap name given above, but you may change it. Just be certain to enter that name in the Helm values, below.

To enable enterprise custom pricing, use the enterpriseCustomPricing values. Enable the feature by setting the enabled value to true. Additionally, make sure that your configMapName matches and location.URI match the ConfigMap and spec file names used previously.

enterpriseCustomPricing:
  enabled: true
  configMapName: kubecost-enterprise-pricing
  location:
    URI: /var/configs/enterprise-pricing/spec.csv

kubecostAggregator:
  useDBv3: true

Note that in version 2.8 you must additionally set kubecostAggregator.useDBv3 to true. In future releases, that setting will become the default.

Above, we saw an example of a pricing spec. Here, we will give detailed instructions on how to create a new pricing spec from scratch.

Begin with the following empty CSV template.

Each row in the CSV will represent pricing for a particular instance type (e.g. “m5.large”, “standard”) of the given asset class (e.g. “node”, “volume”). Below we will provide instruction and examples for how to define pricing and identifying information for each asset class.

Nodes

For example, for a node of InstanceType a4 with 4 CPUs and 8 GiB RAM, the following two methods express the same total hourly price:

Users can identify categories of nodes for pricing according to three methods: InstanceType, InstanceType and Region, or Label. If a node matches multiple rows in the spec, then we have to decide on the priority of the spec rows. The priority, from highest to lowest, is Label, then InstanceType and Region, then InstanceType.

Given the above example, a node that matches all three of the spec rows will be assigned an hourly price of 0.28, as Label is the strongest match.

GPUs

For any given GPU, the priority is to use Label first, then InstanceType. So, given the above example, a GPU that matches both of the spec rows will be assigned an hourly price of 4.120, as Label is the strongest match.

Volumes

Pricing volumes with the GB-hour unit means that the total resultant cost of a volume will be computed as VolumeCapacity * Hours * PricePerUnit, e.g., a 100 GB volume that runs for 24 hours at a price-per-unit of 0.008 will cost 100.0 * 24.0 * 0.008 = 19.20.For any given volume, the priority is to use Label first, then InstanceType and Region, then InstanceType. So, given the above example, a volume that matches all three of the spec rows will be assigned a per-GB hourly price of 0.008, as Label is the strongest match.

Load Balancers

Read the following section for some troubleshooting information.

It is possible to enable Enterprise Custom Pricing, but to have an invalid or unparsable pricing spec. In that case, Kubecost will not utilize custom pricing. To check if this is the case, you can look on the Settings page, under Enterprise Custom Pricing, and you can look for error logs.

A successful Kubecost installation with Enterprise Custom Pricing enabled will look as below.

The logs for a successful install will show the following.

INF enterprise custom pricing: enabled, will load from CSV file at '/var/configs/enterprise-pricing/pricing.csv'
INF enterprise custom pricing: successfully loaded CSV
INF enterprise custom pricing: clearing pricing records not matching spec checksum 0d567e56e35a109534a3629ae2d5da81

Missing Enterprise Key

Enterprise Custom Pricing is an enterprise feature, so deploying with the feature enabled, but without a valid enterprise key, will lock the application.

To enable Enterprise Custom Pricing, be sure to sign up for an enterprise contract and include your product key during installation.

Invalid Pricing Spec.

If your pricing spec is invalid, you will see warning messages on the Settings page.

You will also see logs, reflecting the same information.

INF enterprise custom pricing: enabled, will load from CSV file at '/var/configs/enterprise-pricing/pricing.csv'
INF enterprise custom pricing: successfully loaded CSV
ERR enterprise custom pricing: pricing spec is invalid with 2 issues: 
line 9: invalid node pricing: node pricing must use total pricing (unit 'hour') or resource pricing (units 'cpucorehour' and 'ramgbhour')
line 10: invalid node pricing: node pricing must use total pricing (unit 'hour') or resource pricing (units 'cpucorehour' and 'ramgbhour')

Unparsable Pricing Spec.

It may be the case that the pricing spec cannot be parsed at all. In that case, you will see an error on the Settings page.

You will also see logs, reflecting the same errors.

INF enterprise custom pricing: enabled, will load from CSV file at '/var/configs/enterprise-pricing/pricing.csv'
ERR enterprise custom pricing: failed to load CSV: failed to parse file '/var/configs/enterprise-pricing/pricing.csv': error reading pricing spec CSV file '/var/configs/enterprise-pricing/pricing.csv': record on line 2: wrong number of fields

1. Create a CSV file in this format (also in the below table). CSV changes are picked up hourly by default.

   1. EndTimeStamp: currently unused
   2. InstanceID: identifier used to match asset
   3. Region: filter match based on topology.kubernetes.io/region
   4. AssetClass: node pv, gpu are supported
   5. InstanceIDField: field in spec or metadata that will contain the relevant InstanceID. For nodes, often spec.providerID , for pv’s often metadata.name
   6. InstanceType: optional field to define the asset type, e.g. m5.12xlarge
   7. MarketPriceHourly: hourly price to charge this asset
   8. Version: field for schema version, currently unused

If the node label topology.kubernetes.io/region is present, it must also be in the Region column.

1. The node the GPU is attached to must be matched by a CSV node price. Typically this will be matched on instance type (node.kubernetes.io/instance-type)
2. Supported GPU labels are currently:
   • gpu.nvidia.com/class
   • nvidia.com/gpu_type
3. Verification:
   1. Connect to the Kubecost Prometheus: kubectl port-forward --namespace kubecost services/kubecost-cost-analyzer 9090:9090
   2. Run the following query: curl localhost:9090/model/prometheusQuery?query=node_gpu_hourly_cost
      1. You should see output similar to this: {instance="ip-192-168-34-166.us-east-2.compute.internal",instance_type="test.xlarge",node="ip-192-168-34-166.us-east-2.compute.internal",provider_id="aws:///us-east-2b/i-055274d3576800444",region="us-east-2"} 10 | YOUR_HOURLY_COST

Provide a file path for your CSV pricing data in your values.yaml . This path can reference a local PV or an S3 bucket.

pricingCsv:
  enabled: true
  location:
    provider: "AWS|GCP"
    region: "us-east-1"
    URI: s3://YOUR_BUCKET/path/custom-pricing.csv
    csvAccessCredentials: pricing-schema-access-secret

Alternatively, mount a ConfigMap with the CSV:

kubectl create configmap csv-pricing --from-file custom-pricing.csv

Then set the following Helm values:

pricingCsv:
  enabled: true
  location:
    URI: /var/kubecost-csv/custom-pricing.csv
    csvAccessCredentials: ""

extraVolumes:
- name: kubecost-csv
  configMap:
    name: csv-pricing

extraVolumeMounts:
- name: kubecost-csv
  mountPath: /var/kubecost-csv

For S3 locations, provide file access. Required IAM permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:Get*",
        "s3:List*"
      ],
      "Resource": [
        "arn:aws:s3::: /*",
        "arn:aws:s3::: "
      ]
    }
  ]
}

There are two options for adding the credentials to the Kubecost pod:

1. Service key: Create an S3 service key with the permissions above, then add its ID and access key as a K8s secret:
   1. kubectl create secret generic pricing-schema-access-secret -n kubecost --from-literal=AWS_ACCESS_KEY_ID=id --from-literal=AWS_SECRET_ACCESS_KEY=key
   2. The name of this secret should be the same as csvAccessCredentials in values.yaml above
2. AWS IAM (IRSA) service account annotation

Negotiated discounts are applied after cost metrics are written to Prometheus. Discounts will apply to all node pricing data, including pricing data read directly from the custom provider CSV pipeline. Additionally, all discounts can be updated at any time and changes are applied retroactively.

The following logic is used to match node prices accurately:

• First, search for an exact match in the CSV pipeline
• If an exact match is not available, search for an existing CSV data point that matches region, instanceType, and AssetClass
• If neither is available, fall back to pricing estimates

You can check a summary of the number of nodes that have matched with the CSV by visiting /model/pricingSourceCounts. The response is a JSON object of the form:

{
  "code": 200,
  "status": "success",
  "data": {
    "TotalNodes": 10,
    "PricingType": {
      "csvExact": 5, // exact matches by the providerID field
      "csvClass": 4, // matches where the region and instanceType match
      "": 1 // matches that use our default pricing
    }
  }
}