Logging mbatchd and mbschd profiling information

LSF provides a feature to log profiling information for the mbatchd and mbschd daemons to track the time that the daemons spend on key functions. This can assist IBM Support with diagnosing daemon performance problems.

To enable daemon profiling with the default settings, edit the lsf.conf file, then specify LSB_PROFILE_MBD=Y for the mbatchd daemon or specify LSB_PROFILE_SCH=Y for the mbschd daemon. You can also add keywords within these parameters to further customize the daemon profilers.

The profile dump includes one line for each tracked function call stack, and for each thread. The format of the profile dump is as follows:

PROFILE "functionName": count rt_min rt_max rt_avg rt_total ut_min ut_max ut_avg ut_total st_min st_max st_avg st_total start_time end_time thread_id stack_depth "stack"

The following is a description of each field:

  • functionName: The name of the tracked function.
  • count: The number of times that the function is invoked within the stack corresponding to this record, within the last sampling interval.
  • rt_min: The minimum runtime of one call to the function in this sampling interval.
  • rt_max: The maximum runtime of one call to the function in this sampling interval.
  • rt_avg: The average runtime of the calls to the function in this sampling interval.
  • rt_total: The total runtime of all the calls to the function in this sampling interval.
  • ut_min: The minimum user mode CPU time of one call to the function in this sampling interval.
  • ut_max: The maximum user mode CPU time of one call to the function in this sampling interval.
  • ut_avg: The average user mode CPU time of the calls to the function in this sampling interval.
  • ut_total: The total user mode CPU time of all the calls to the function in this sampling interval.
  • st_min: The minimum system mode CPU time of one call to the function in this sampling interval.
  • st_max: The maximum system mode CPU time of one call to the function in this sampling interval.
  • st_avg: The average system mode CPU time of the calls to the function in this sampling interval.
  • st_total: The total system mode CPU time of all the calls to the function in this sampling interval.
  • start_time: The start time of this sampling interval in UTC time.
  • end_time: The end time of this sampling interval in UTC time.
  • thread_id: The thread ID.
  • stack_depth: The depth of the stack.
  • stack: The stack trace. This is a list of function names separated by # characters.

All time values are in milliseconds.