Collections
The Collections page allows users to create groups of external cloud costs and Kubernetes objects in order to receive a unified view of all cloud spending. This is to leverage existing tags applied to cloud resources, without needing to rename them to match Kubernetes names. Collections will also avoid duplicate costs.
Kubernetes costs, also referred to as in-cluster costs, refer to spending on resources including nodes, disks, and network costs. They are monitored via the Assets dashboard. Cloud costs, also referred to as out-of-cluster or external costs, refer to spending on third party services such as services offered by cloud service providers. They are monitored via the Cloud Cost Explorer.
Creating a new collection
To begin, select Add a new Collection in the top right. If you have not created any collections yet, you should see a Add a Collection icon in the middle of your screen. You can select this instead. The ‘New Collection’ page opens. The headline for your page should be an auto-generated title. This will be the name of your collection, which you can change by selecting the pencil icon. You can provide a name and a category for your collection. Categories are custom labels you can add to multiple collections to allow for filtering of that category, if multiple collections are intended for the same team or function. A collection can have at most one category. Name and category can always be edited later.
Once you are satisfied with the name of your collection, you can add Kubernetes and cloud costs.
All Kubernetes and cloud spend sources will appear on this page under one of two columns: Costs in Collection, or Costs Not in Collection. When first creating your collection, there should be no costs already added. Therefore, you should select Costs Not in Collection to view a list of all available cost sources, or select Explore Costs.
Costs are then divided into two categories: Kubernetes and Cloud (see above for an explanation of these costs). You can toggle between the two groups using the Domain dropdown on the left of the page. For each group, you will see a total cost value displayed on the right of the page, above your cost table.
Costs are organized in a table, listed in descending order starting with the highest values. To add an individual item, hover your cursor over the item until you see a green Add button visible on the right of the page. Select Add to include that cost into your collection. You can add all your listed Kubernetes or cloud costs into a collection by selecting Add All, displayed next to your total cost. This is only available once you've added at least one filter. Once an item has been added to the collection, changes will be automatically saved.
You can filter and recategorize your cost table using Aggregate By and Add Filters. Aggregate By allows you to organize your costs by a listed category, and supports single and multi-aggregation. Add Filters allows for flexible filtering of your table items, including filtering by custom labels.
After having added items to your collection, selecting Costs in Collection will provide a complete list of all items, as well as key cost metrics including total and percentage costs of both Kubernetes and cloud items.
In the event there is cost overlap from conflicting Kubernetes and cloud costs, they will be reflected in a special Overlap category. Kubecost automatically subtracts the overlapping costs so that the totals seen for the collection are accurate and do not contain duplicate costs. Currently, overlap is not considered for shared costs and Load Balancer costs coming from workloads running on Azure and GCP clusters.
Overlap may change as costs in Collections are reconciled. We recommend adjusting the query window to point to reconciled data only; typically, this means excluding the last 48 hours. See more on reconciliation here.
Managing collections
The Collections page will list all existing collections with a chart displaying cost over time and total cost. You can begin editing a collection by selecting it. You can also adjust your Collections display by adjusting the window of time, aggregating by item or kind, or filtering.
Selecting the three horizontal dots in the top right of the collection tile will provide you with additional options:
- Delete
- Export as CSV
- Export as PDF
Viewing idle costs
Costs in the Kubernetes domain have a corresponding idle component. For any Kubernetes costs part of a collection, the idle cost can be optionally configured to be included in the total cost displayed. This can be done from the Settings page under Idle in Collections.
The idle cost can be shared by cluster or by node. We do not recommend sharing idle by cluster as that may lead to cost inconsistencies when deduplicating the Kubernetes costs against the cloud costs for the same resource. To learn more about how costs in Collections are calculated, see this blog post.
By default, idle costs are hidden. If enabled, the 'Idle' column on the Costs in Collection view will display the corresponding idle cost under each item.
To learn more about sharing idle costs, see here.
Categories and Chargeback
Starting in v2.7, the Category view can be used for chargeback reporting. Each collection in the category will be represented as a chargeback line item.
The new costs table will display the following columns:
- Cost: "Complete" collection cost independent of the Category, prior to any overlap deductions
- Overlap: deduction applied to the collection line item in the case when (part of) its cost has already been accounted for in another collection within the same category. See Allocating overlap in collections for more information
- Shared Cost: Total shared cost attributable to collection
- Chargeback cost: Final chargeback cost for collection
Cost Sharing
You can share the costs of one or more existing collections across a category. The costs of the shared collections will be distributed across all the collections in the category.
Start typing a collection name in the 'Find a collection' textbox. The dropdown will be populated with collections as you type.
The costs can be shared in one of two strategies:
- Weighted: costs are distributed proportionally based on the collection's (original) cost
- Even: costs are distributed in equal proportions for each collection in the category
The shared cost attributable to each collection in the category will be added to the collection cost (after any overlap deductions are applied) to render the final Chargeback cost for the collection.
You can add, modify or remove collections from the list of shared collections by clicking _Edit_ in the upper right corner.
Allocating overlap in collections
Given their configuration, there may exist overlap between the collections in the same category. For example, if the Engineering collection covers the costs where the label "team" has value "engineering" and the Finance collection covers the costs for namespace "application", overlap would be represented by the cost of all resources in namespace "application" that are also tagged with label "team":"engineering".
Chargeback costs are calculated for each collection within the context of the category. If there is overlap between collections, the overlapping costs are allocated in order of priority, where the collection with superior priority maintains its full cost, while the one with inferior priority will have a deduction in the Overlap column.
The collections are prioritized in line with the order they appear in on the costs table. By default, collections are prioritized by their Cost (most expensive collections are first). Collections can also be prioritized in alphabetical order by modifying the setting in the dropdown to "Order by Name".
If any shared collections are set, their costs are automatically prioritized respective to the collections that are part of the Category. If there is overlap between the shared collections themselves, it will be handled in the same manner as described above. The priority can be modified by reordering the shared collections using the arrows on the right side of each line item.
Managing categories
The Category page will list all the collections in the category with a chart displaying cost over time and total cost. You can begin editing a collection by selecting it. You can also adjust your Category display by adjusting the window of time.
Selecting the three horizontal dots next to the priority dropdown will provide you with additional options:
- Export as CSV
- Export as PDF