Microsoft Azure and other providers configuration

1. Open the Costing Standard project.
2. Click the Projects tab.
3. Click Components in the ribbon.
4. Click the CTF - Azure component.
5. Click Install.

To bring the cloud billing files into the Costing Standard application, upload the data into a new table. You can upload data manually as described below, or you can upload data using the Apptio DataLink application. We recommend using the DataLink application.

• Upload data using DataLink

  Use the references below to find information on using DataLink connectors to upload data from the cloud service providers. Look for the guides for the specific cloud service such as AWS and Azure.

  • DataLink 3: see the DataLink 3 Knowledge Base in the Apptio Community.
  • DataLink 4: see the DataLink 4 Knowledge Base in the Apptio Community.
• Upload a cloud billing file manually
  1. Click the Home tab in the Ribbon.
  2. Click the New menu and then click Table.
  3. In the Create Table dialog, enter a name for the table. Use any name that fits with your workflow.
  4. Enter a category. As you start typing, matching names of categories that already exist will be displayed. Also, you can enter a new category name. The categories are used to organize the tables in the Project Explorer.
  5. Click OK. The application creates the table and displays the Source step in the transform pipeline as shown below.

     

  6. Click the File Upload option.
  7. Do one of the following to add an Import step to the pipeline:
     • Click in the Details pane and browse to the file you want to upload.
     • Drag a file into the Details pane.
  8. Click the Import step in the pipeline and review the settings:
     • Start import at row - Indicate the first row in the table to be included in the import.
     • Columns to Exclude - Indicate if there are columns you want to exclude.
     • Text Encoding - If the data file uses a particular character encoding, select the encoding scheme from the list. If you are unsure about the encoding, select the Autodetect encoding option.
     • Delimited - Select the character that separates the data fields in the file. Most often this will be a comma.
     • Text qualifier - If there are special text characters in the data, indicate the text qualifier that is used to surround those characters.
     • Columns - Review the columns listed at the right, and if desired, change the column types. To filter by column type (key, text, number, date), click a type icon in the search field of the Type column.
  9. To review the uploaded table, click the Table step in the pipeline.
  10. If the table is acceptable, click Save in the Ribbon.
  11. If you are done making edits, click Check In in the Ribbon.

The bills from cloud providers include a rich set of data. To take full advantage of this data in the Costing Standard application, you will need to transform the data to match the Cloud Service Provider Master Data table. To transform the data, you will add columns to the table to match your data to the Cloud Service Provider Master Data table. The columns you add will depend on your data and whether you are using AWS or Azure data.

• Add a column.
  1. If you have not done so already, add a Formulas step to the data transform pipeline. To add a Formula step, move the cursor to the line dividing two existing steps in the pipeline and then click the plus sign at the right edge of the Transform Steps panel.
  2. In the Details area, click Formulas.
  3. Click Add a new column.
  4. If needed, change the column type by clicking the Type field.
  5. Enter a name for the column.
  6. Enter a formula for the column.
  7. Click Save in the Ribbon.
• Add descriptive columns.

  The fields listed below are included in the Cloud Service Provider Master Data table. The fields are used to create slicers on the Costing Standard cloud reports. The cloud data you import may contain fields (e.g: resource tags) that correspond to these fields. To make it easier to map the data to the matching fields in the Cloud Service Provider Master data set, we recommend that you create the fields below in the billing data table.

  • Application
  • Environment
  • Purpose
  • System Owner
  • Project
  • Cost Center

  For example, assume you have a field in the billing table called Business Owner. You might want to map that field to the System Owner field in the Cloud Service Provider Master table. In the billing data table, you would create a field called System Owner and set that field equal to Business Owner. Then when you append the billing data table to the Cloud Service Provider Master table, the System Owner fields will be mapped automatically.

Think of the Cloud Service Provider Master Data table as the central repository for all of the cloud data. As you bring data into the application each month, you will add it to this table by "appending" the data. If you are using a DataLink connector, the data automatically will be brought in each month.

1. In the Project Explorer, click the Cloud Service Provider Master Data table.
2. Click the Append step in the pipeline.
3. Click Append Table in the Details area.The Append to... dialog is displayed as shown below.
   
4. Select the transform data set and click Next.The application displays the mapping dialog shown below.
   
5. Map the columns as shown in the tables in the mapping sections below.If a field is not available from the drop-down list, type the value in the field. As you begin entering the text, matching column names will be displayed. Note when entering " && ", there is a space before and after the &&. All of the fields listed below must be mapped.
6. Click Save.

   Table 1. Cloud service provider fields

   Field	Short Description	Formula
   Item	The second level categorization of the service provided (e.g. for AWS, "Box Usage").
   Lookup	An autogenerated concatenation of the Item and Procedure columns; this is used for the service mapping lookup from the Cloud Service Provider Lookup that maps the specific provider service to the IT Resource Sub-Tower.	=Item && Procedure
   Procedure	The third level categorization of the service provided (e.g. for AWS, the operation the Type / usage is preforming -- RunInstances:0002)
   Type	The generic grouping for the item used by the application.
   Unit	The unit of measure for the service provided.
   Cost	The total, net cost of the service to the customer (inclusive of any tax, discounts or other factors).
   Description	The detailed description of the service provided.
   ITRT_Cloud_Key	A predefined key used to allocate costs from the Cloud Service Provider object to the IT Resource Towers object.	="Public_Cloud" && Subtower
   Product	Name of the cloud service provider. NOTE: For allocations to flow from the Vendor object, this name must match the company name in the Vendors list.
   Public Cloud Metadata 1	Additional information from the bill you would like to report on.
   Public Cloud Metadata 2	Additional information from the bill you would like to report on.
   Public Cloud Metadata 3	Additional information from the bill you would like to report on.
   Rate	The effective average unit cost (or rate) of the service consumed; calculated by dividing cost by quantity.	= Cost / Usage Qty
   Subtower	The mapped IT Resource Sub-Tower to which the service belongs (e.g. for AWS, "Box Usage" is mapped to Cloud Compute and, based on the Procedure, it is mapped to Cloud Compute Linux or Cloud Compute Windows).	=Lookup(Lookup,Cloud Service Provider Lookup,Lookup,SubTower)
   Type	The generic grouping for the item used by the application.
   UID Metafield	Prescriptive unique identifier used to determine the level of granularity of the line items in the provider bill.
   Unique ID	A unique identifier or generated concatenation of columns to establish the desired granularity of data for allocations and reporting.	=Customer Number && Provider && Product && Item && Procedure && UID Metafield
   Unit	The unit of measure for the service provided.	=Lookup(Lookup,Cloud Service Provider Lookup,Lookup,Unit)
   Usage Qty	The quantity or total amount of the service delivered.
   Vendor_CSP Key	A predefined key used to allocate costs from the Vendor object to the Cloud Service Provider object.	="vendor_cloud" && Provider
   Application	The application consuming the cloud service.
   Environment	The stage in the development lifecycle where the infrastructure is being used. Apptio Required values include: Production, Development, Test, DR
   Purpose	The general use for the cloud resource, such as Web Server, App Server, Database Server, Utility Server, etc.
   Consumer	The primary business consumer of the cloud service.
   Consumer ID	The ID of the primary business consumer of the cloud service.
   System Owner	The individual responsible for maintaining the cloud service.
   Project	The name or ID of the project for which the cloud service is used.
   Cost Center	The specific cost center responsible for budgeting and managing cloud service.

   Table 2. Azure mapping

   Cloud Service Provider Data	Microsoft Azure Billing Transform
   Item	=Meter Category&&Meter Sub-Category
   Procedure	=Meter Name
   Cost	=ExtendedCost
   Description	=Product
   Product	=Meter Category
   Provider	=”Microsoft Azure”
   Usage Qty	=BillableResourceQty
   UID Metafield	Customer-specific. This field should include enough customer-specific data (e.g. tags) in order to ensure the Unique ID field is unique. See Unique ID Reference for more details.
   Application	Customer specific. Use account/subscription or tags to populate.
   Environment	Customer specific. Use account/subscription or tags to populate.
   Purpose	Customer specific. Use account/subscription or tags to populate.
   Consumer	Customer specific. Use account/subscription or tags to populate.
   Consumer ID	Customer specific. Use account/subscription or tags to populate.
   System Owner	Customer specific. Use account/subscription or tags to populate.
   Project	Customer specific. Use account/subscription or tags to populate.
   Cost	Customer specific. Use account/subscription or tags to populate.

1. Update the Service Mapping files in your environment.
2. For AWS bills uploaded with DataLink, make sure you check the Upload latest Apptio AWS Services mapping data option.
3. For Azure bills or AWS bills not uploaded with DataLink, update the mapping file with the latest version available on Apptio Community. Go to Apptio Community on TBM Connect (https://community.apptio.com) and search for Services Mapping.
4. If there are still gaps in the mappings, you can create an Excel file and append it in to the Services Mapping master data set. See steps 7 – 9.

The CTF application uses sub-towers to correctly allocate cost. The out-of-the-box web services data may not have contained enough information to assign a sub-tower to each entry in the file. If there are entries without assigned sub-towers, you must create an Excel file that provides the necessary sub-tower information. In later steps, you will upload the Excel file and map it to the Cloud Service Provider Lookup Master Data data set.

1. Create an Excel file with a name that matches your work flow.
2. Add the following columns to the Excel file:
   • Item (usage type)
   • SubTower
   • Type (based on the billing data)
   • Unit
   • Procedure
3. Enter information in the columns for each line in the web services file that does not have an automatically assigned sub-tower.
4. Save the spreadsheet as a .csv file.

To bring the Excel file you created in the previous step into the Costing Standard application, upload the data into a new table

1. Click the Home tab in the Ribbon.
2. Click the New menu and then click Table.
3. In the Create Table dialog, enter a name for the table.
4. Enter a category.

   As you start typing, matching names of categories that already exist will be displayed. Also, you can enter a new category name. The categories are used to organize the tables in the Project Explorer.

5. Click OK.

   The application creates the table and displays the Source step in the transform pipeline as shown in the following image.

   
6. Click the File Upload option.
7. Do one of the following to add an Import step the pipeline:
   • Click in the Details pane and browse to the file you want to upload.
   • Drag a file into the Details pane.

   An Import step is added to the pipeline.

8. Click the Import step in the pipeline and review the settings:
   • Start import at row: Indicate the first row in the table to be included in the import.
   • Columns to Exclude: Indicate if there are columns you want to exclude.
   • Text Encoding: If the data file uses a particular character encoding, select the encoding scheme from the list. If you are unsure about the encoding, select the Autodetect encoding option.
   • Delimited: Select the character that separates the data fields in the file. Most often this will be a comma.
   • Text qualifier: If there are special text characters in the data, indicate the text qualifier that is used to surround those characters.
   • Columns: Review the columns listed at the right, and if desired, change the column types. To filter by column type (key, text, number, date), click a type icon in the search field of the Type column.
9. To review the uploaded table, click the Table step in the pipeline.
10. If the table is acceptable, click Save in the Ribbon.
11. If you are done making edits, click Check In in the Ribbon.

After uploading the sub-towers Excel data table, the next step is to append the table to the Cloud Service Provider Lookup Master Data table. After you append the table, the entries in the billing transform table that were missing a sub-tower entry should now have entries.

1. In the Project Explorer, click the Cloud Service Provider Lookup Master Data table.
2. Click the Append step in the pipeline.
3. Click Append Table in the Details area.
4. Click the Excel table and click Next.
5. Map the columns as shown in the tables in the mapping sections below.

   

   If a field is not available from the drop-down list, type the value in the field. As you begin entering the text, matching column names will be displayed. Note when entering " && ", there is a space before and after the &&. All of the fields listed below must be mapped.

6. Click Save.

If you have chosen to not install the Vendors component, you must manually configure the allocation from the Cost Source object to the Cloud Service Provider object in the Cost model. After installing the Cloud Service Provider component, follow the steps below to configure the allocation

1. In uploaded cost source data set, add a new column:
   • Name: Cost Source_Cloud Key
   • Type: Key
   • Value: ="Vendor_CSP"&&Vendor Name
2. Save the table.
3. In the Project Explorer, click the Cost Source Master Data table.
4. Click the Append step in the transform pipeline.
5. Edit the appended cost source table.
6. In the mapping table, click Select additional source columns.
7. Select your new column Cost Source_Cloud Key and click OK.
8. In the Project Explorer, click the Cloud Source model table.
9. Under Allocations, click Add Allocation.
10. Complete the fields to allocate value to the Cloud Service Provider object.