Creating the user-defined metrics dashboard and SQL processing

Create Grafana dashboards and the associated SQL procedures to view and analyze user-defined metrics.

Procedure

  1. Log in to Grafana.
  2. To create a Grafana dashboard, you can copy an existing dashboard and modify the copy to suit your needs:
    1. Click the share icon, which is next to the dashboard name at the top of the dashboard.
    2. Click Export > View JSON.
    3. Copy the whole JSON text to an editor.
    4. Edit the JSON text to suit your needs.
    5. In Grafana, click the plus (+) icon on the left, and click Import to import the JSON file back to Grafana.
    6. Refine the dashboard.
    7. Click Export > View JSON to save the dashboard.
    If you want to create a dashboard from scratch, at least include variables like system and metrics and use some of the existing stored procedures.
  3. Ensure that your user-defined metrics data is in the database.
  4. Implement the variables that you need at the top of the dashboard.
  5. After the variables are working, use those variables as the input to implement the table, graph, or other panels.
    Before you implement the table, graph, or other panels, you can test the stored procedures by using a database tool. After the panels are functioning, refine the presentation of the data on the Grafana UI.
  6. Provision your dashboards:
    1. Create a file like tpf_data_sci/user_files/tpf_sample_user_defined_metrics/grafana/udm.yaml to specify the directory where the dashboard JSON source file exists and the directory that the dashboard will be created in.
    2. Copy the dashboard JSON source file and .yaml file to the tpf_data_sci/Docker/tpf_grafana_docker_files/provisioning/dashboards directory by using a script.
      You can modify or create a script based on the tpf_data_sci/Docker/tpf_prepare_configurations.sh script, which contains the code that copies the sample user-defined metrics dashboard files from the tpf_data_sci/user_files/tpf_sample_user_defined_metrics/grafana directory to the tpf_data_sci/Docker/tpf_grafana_docker_files/dashboards directory.
    3. Rebuild your Grafana container.
    This procedure provides only one of the possible ways to provision your dashboards. You also can implement a different mechanism.
  7. Develop your SQL procedures to view your data in the dashboard:
    The tpf_data_sci/user_files/tpf_sample_user_defined_metrics/database/tpf_sample_udm_stored_procedures.sql file shows an example. For more information about how this file is administered in the sample analytics pipeline, see Creating user-defined metrics table definitions. The following SQL stored procedures are implemented in the sample user-defined metrics:
    tpf_sample_udm_data_types_available
    This stored procedure, which is called by the Data Type variable on the sample dashboard, checks the data in udm_metrics_table for all of the selected systems in the specified time frame, and returns all data_type values that are used in that time range. This procedure returns a table of data_type and id values. The id values are used for subsequent calls in the dashboard.
    tpf_sample_udm_group_names_available
    This stored procedure, which is called by the Group Name variable on the sample dashboard, checks the data in udm_metrics_table for all of the selected systems in the specified time frame, and returns all group_name values that are used in that time range. This procedure returns a table of group_name and id values. The id values are used for subsequent calls in the dashboard.
    tpf_get_udm_metrics
    This function is used by various stored procedures that retrieve the available system ID, metric types, metrics, and so on. Search the sample analytics pipeline code base for the use of this function. The JSON definition in this function provides the available metric types and metrics that are displayed in the variables in the sample dashboard.
    tpf_user_defined_metrics_graph
    This stored procedure produces metrics for display in the graph panel for the variable values that are selected on the sample dashboard. Examine how this procedure can be used to get averages or totals of metrics across systems.