Monitoring collector examples

These examples show how metrics are collected in the plugin.com.ibm.was plug-in.

Script collector

The plugin.com.ibm.was plug-in can use the script collector to run its shell scripts to collect the metrics from wsadmin. The following files show use of the script collector.

monitoring_register_collector.py

This file is the collector registration script that is in the plugin/parts/was.scripts/WAS/installApp_post_handlers subdirectory. It creates the registration data and configuration for the script collector and implements registration of the collector by using maestro.monitorAgent.register. The metadata file and executable script file are placed in predefined directories in the parts subdirectory and are copied into corresponding directories when the plug-in is installed on a virtual machine. In this example, the paths of the metadata file and executable script are concatenated from those maestro variable values and relative path of the predefined directories. The following code that was extracted from monitoring_register_collector.py shows how the config in the registration is built.

node = maestro.node
role = maestro.role
…
scriptdir = node['scriptdir']
node_name = node['name']
role_name = node['name'] + '.' + role['name']
monitorscripts = scriptdir + '/WAS/monitor_scripts/'
metafile = monitorscripts + '/metadata/WASModuleMetrics.metadata'
executable = monitorscripts + '/wsadmin_controller.sh'
registration = '{ "node":"' + node_name + '","role":"' + role_name 
               + '","collector" : "com.ibm.maestro.monitor.collector.script","config":                      
               {"metafile":"' + metafile + '","executable":"' 
               + executable + '","arguments": "","validRC": "0","workdir":"' 
               + monitorscripts + '","timeout": "10"}}'

The registration data and config looks like the following JSON object when this script runs:

{
    "node": "Web_Application-was.11310621328463",
    "role": "Web_Application-was.11310621328463.WAS",
    "collector": "com.ibm.maestro.monitor.collector.script",
    "config": {
        "metafile": "/opt/IBM/maestro/agent/usr/servers
                     /Web_Application-was.11310621328463/monitor
                     /wascollector/WASModuleMetrics.metadata",
        "executable": "/opt/IBM/maestro/agent/usr/servers
                       /Web_Application-was.11310621328463
                       /monitor/wascollector/wsadmin_controller.sh",
        "arguments": "",
        "validRC": "0",
        "workdir": "/opt/IBM/maestro/agent/usr/servers
                    /Web_Application-was.11310621328463/monior/wascollector",
        "timeout": "10" 
    }
}

The script wsadmin_controller.sh calls other scripts that include run_wsadmin.sh and collector_agent.jy, which starts wsadmin, formats data from wsadmin into a local file and reads the data from the file, and then prints metrics. The implementation of all these scripts depends mainly on wsadmin usage.

WASModuleMetrics.metadata

This metadata file is in the plugin/parts/was.scripts/WAS/monitoring_scripts subdirectory. It defines metric categories and data types.

{
    "Version" : 1,
    "Category": [
        "WAS_WebApplications",
        "WAS_JVMRuntime",
        "WAS_TransactionManager",
        "WAS_JDBCConnectionPools"
    ],
    "Metadata": [
        {
            "WAS_WebApplications" : {
                "updateInterval": 15,
                "metrics": [
                    {
                        "attributeName": "MaxServiceTime",
                        "metricName": "MaxServiceTime",
                        "metricType": "RANGE"
                    } ,
                    {
                        "attributeName": "ServiceTime",
                        "metricName": "ServiceTime",
                        "metricType": "RANGE"
                    } ,
                    {
                        "attributeName": "RequestCount",
                        "metricName": "RequestCount",
                        "metricType": "COUNTER"
                    } ,
                    {
                        "attributeName": "MinServiceTime",
                        "metricName": "MinServiceTime",
                        "metricType": "RANGE"
                    }
                ]
            }
        },
…… ……

monitoring_ui.json

This presentation metadata file is in the plugin/parts/templates subdirectory. It defines how metrics are displayed.

[
    {
        "category": "WAS_WebApplications",
        "label": "WAS_WebApplications_Request_Count",
        "displays": [
            {
                "label": "",
                "monitorType": "HistoricalNumber",
                "chartType": "Lines",
                "metrics": [
                    {
                        "attributeName": "RequestCount",
                        "label": "Request_Count" 
                    } 
                ] 
            } 
        ] 
    },
    {
        "category": "WAS_WebApplications",
        "label": "WAS_WebApplications_Service_Time",
        "displays": [
            {
                "label": "",
                "monitorType": "HistoricalNumber",
                "chartType": "Lines",
                "metrics": [
                    {
                        "attributeName": "MaxServiceTime",
                        "label": "Max_Service_Time" 
                    },
                    {
                        "attributeName": "MinServiceTime",
                        "label": "Min_Service_Time" 
                    },
                    {
                        "attributeName": "ServiceTime",
                        "label": "Avg_Service_Time" 
                    } 
                ] 
            } 
        ] 
},
…… ……

HTTP collector

The plugin.com.ibm.was plug-in can also use HTTP collector to send a request to the PerfServletApp application that is running on WebSphere® Application Server and get metrics by processing the HTTP response. PerfServletApp is a default web application that is installed with WebSphere Application Server. It provides a way to work with the Performance Monitoring Infrastructure (PMI) to retrieve original performance statistics. If the application is not included with the WebSphere Application Server installation, it must be included as a part of plugin.com.ibm.was plug-in and it must be installed before you register and use the HTTP collector. The following files from the plug-in show the use of the HTTP collector.

monitoring_register_http_collector.py

This script functions like monitoring_register_collector.py described in the script collector example, except for making config for an HTTP collector type. It is stored in plugin/parts/was.scripts/WAS/installApp_post_handlers. The PerfServletApp application returns performance statistics from PMI in XML format, which is not compatible with the JSON format used by the monitoring collectors. A utility .jar called WASResponse.jar is used to transform the XML data to the required JSON format. This file is placed under parts/was.scripts/WAS/monitoring_scripts with other scripts and metadata files. The registration includes the datahandler properties in the config. The following example shows how the config in registration is built.

node = maestro.node
role = maestro.role
…
scriptdir = node['scriptdir']
node_name = node['name']
role_name = node['name'] + '.' + role['name']
monitorscripts = scriptdir + '/WAS/monitor_scripts/'
metafile = monitorscripts + '/metadata/WASModuleMetrics.metadata'
url = 'http://localhost:9080/wasPerfTool/servlet/perfservlet'
jardir = scriptdir + '/WAS/monitor_scripts/handler/'
registration = '{ "node":"' + node_name + '","role":"' + role_name + '",
               "collector" : "com.ibm.maestro.monitor.collector.http",
               "config":{"metafile":"' + metafile + '","url":"' + url + '",
                   "datahandler": {"jardir":" '+ jardir + '",
                           "class" :"com.ibm.maestro.was.perf.formatter.Transformer",
                           "method":"PMIStatToIWDMetrics" }}}'

The registration data and config looks like the following JSON object when this script runs:

{
    "node": "Web_Application-was.11310621328463",
    "role": "Web_Application-was.11310621328463.WAS",
    "collector": "com.ibm.maestro.monitor.collector.http",
    "config": {
        "metafile": "/opt/IBM/maestro/../WASPerfMetadata.json",
        "url": "http://localhost:9080/wasPerfTool/servlet/perfservlet",
        "query": "",
        "datahandler": {
            "jardir": "resources/jars",
            "class": " com.ibm.maestro.was.perf.formatter.Transformer ",
            "method": " PMIStatToIWDMetrics"
        }
    }
}

In this example, datahandler indicates the directory that contains WASResponse.jar in jardir and the com.ibm.maestro.was.perf.formatter.Transformer is the name of the class in the utility .jar which is exposing the public method named PMIStatToIWDMetrics.

monitoring_install_perservlet.py

To install the PerfServletApp application, the script monitoring_install_perservlet.py is included in the parts/was.scripts/WAS/before_start_handlers subdirectory which is called by default right before WebSphere Application Server is started and after its installation is finished. At execution time, wsadmin is already available to use to install and start PerfServletApp. It uses the maestro interface to install the servlet application. The script also provides PerfServletApp information to wsadmin for finding the servlet package. The following example is an extract of the script.

cmd = [installDir + '/IBM/WebSphere/AppServer/bin/wsadmin.sh', '-conntype', 'None', '-lang', 'jython', ]
archive_file = scriptdir + '/WAS/monitor_scripts/PerfServletApp.ear'
archive_name = 'PerfServletApp.ear'
args = [archive_file, archive_name]
cmd.extend(['-f', scriptdir + '/WAS/was_install_app.py'])
cmd.extend(args)

rc = maestro.trace_call(logger, cmd)
maestro.check_status(rc, 'Installing WAS performance servlet')

The metadata files WASModuleMetrics.metadata and monitoring_ui.json in the script collector example can also be used for the HTTP collector.