Customizing the Python data collector

You can set the variables to change the default behavior of the Python data collector.

User-defined environment variables for the Python data collector

For Python monitoring in IBM® Cloud Private or OpenShift, set the following variables in the application yaml file.

For on-premises applications, set the following variables as environment variables or in the config.properties file.

Table 1. Supported user-defined environment variables for Python monitoring
Variable name Value Description
KPG_LOG_LEVEL
  • DEBUG
  • ERROR
  • INFO
  • Warning
  • DEBUG: Only useful debug information is printed in the log, for example, collected data, data that are sent to server, and server response.
  • ERROR: Only information about exceptions and unexpected situations is printed in the log.
  • INFO: The summary information about the data collector for the user to know what it is doing is printed in the log.
  • Warning: Warning information is printed in the log.
KPG_LOG_TOCONSOLE True The log is printed to console and you can see the log by running the command kubectl logs <pod name>:.
Tip:
  • If you do not set KPG_LOG_TOCONSOLE to True, two log files kpg_dc.log and kpg_restclient.log are created to record the log messages in a sub folder kpg_logs that is in your Python application root path.
  • For Python runtime, standard output(stdout) and standard error(stderr) are buffered. If messages are written into stdout or stderr, they might not be flushed in time if buffer is not full. In this case, when exceptions are reported from Python runtime or Django framework, you might not find any information with the command kubectl logs or the log files kpg_dc.log and kpg_restclient.log. To solve this issue, you can disable stdout and stderr buffering by setting environment PYTHONUNBUFFERED to 1 in the application yaml file.
KPG_GC_STATS True All statistical functions of python garbage collection are enabled. When you set this value to True, it equals running the following command:
  • For Python 2.7, 
    
gc.set_debug(gc.DEBUG_STATS |
gc.DEBUG_COLLECTABLE |
gc.DEBUG_UNCOLLECTABLE | 
gc.DEBUG_INSTANCES | 
gc.DEBUG_OBJECTS )
  • For Python 3.6.x and 3.7.x, 
    
gc.set_debug(gc.DEBUG_STATS |
gc.DEBUG_COLLECTABLE |
gc.DEBUG_UNCOLLECTABLE

To disable KPG_GC_STATS, delete this environment variable. Do not set it to False.

Note: Never set KPG_SAVE_ALL=True in your formal production environment. It is only for the debug mode. Make sure that enough memory is assigned to the application.
KPG_SAVE_ALL True All unreferenced objects are saved into gc.garbage, and you must clear gc.garbage every minute (the data collector clears it for you). When the value is set to True, it equals running the following command: 
gc.set_debug(gc.SAVE_ALL)
To disable KPG_SAVE_ALL, delete this environment variable. Do not set it to False.
Note: Never set KPG_SAVE_ALL=True in your formal production environment. It is only for the debug mode. Make sure that enough memory is assigned to the application.
KPG_ENABLE_OPENTT "False"

By default, the Python data collector enables OpenTracing function. You can disable OpenTracing by setting the environment variable to false.

Example:
- name: KPG_ENABLE_OPENTT
  value: "false"
OpenTracing sampling:
  • JAEGER_SAMPLER_TYPE
  • JAEGER_SAMPLER_PARAM
 The default sampler type is "probabilistic", and the default sampler param is 0.01, which means that 1 in 100 traces will be sampled. You can set it to other values. For more information, see Sampling. When the OpenTracing function is enabled, you can set the OpenTracing sampler type and param. Example:
- name: JAEGER_SAMPLER_TYPE
  value: "probabilistic"
- name: JAEGER_SAMPLER_PARAM
  value: "0.1"
LATENCY_SAMPLER_PARAM Any value between 0 and 1

The default value is 0.1, which means getting 1 request out of 10 requests. The value must be between 0 and 1. The value of 0 means no latency data will be collected. The value of 1 means no sampler and all requests data will be collected.

Example: 
- name: LATENCY_SAMPLER_PARAM
  value: "0.2"