Data sources for workspaces
Monitoring Agent for WebSphere® Applications (also referred to as WebSphere agent) collects data from several sources on the application server. Different workspaces use data from different sources.
Types of data
The WebSphere agent collects the following four types of data:
- Resource data
- Request data
- Data from WebSphere log files
- Process data from the operating system
These four categories are based on the source from which the monitoring data is collected. The workspaces in the Cloud Pak System Software Monitoring Portal display data from these sources. Usually, each workspace displays data from one of the sources. Some workspaces contain summary information collected from one or more sources and processed by the WebSphere agent .
Resource Data
System Monitoring for WebSphere Applications obtains resource data from the Performance Monitoring Infrastructure (PMI) component that WebSphere Application Server provides. This data mostly contains usage and performance information about a resource in the system. For example, the "DB Connection Pools" workspace provides data about connection pool resources, including how long a connection is checked out, how many threads are waiting for connection, and so on.
| Workspace | Description |
|---|---|
| Web Applications workspace | Displays performance data for each web application (.war). This data includes number of requests, average response time, and number of errors. Note that one Web application may have multiple URLs for user requests and the response times for all the requests are aggregated in this window. |
| Servlets/JSPs - Selected Web Application workspace | Breaks down the performance data for a particular Web application into the servlets or JavaServer pages (JSPs) that it contains. Again, each servlet/JSP can contain or be mapped to multiple URLs and all user requests are combined. |
| Sessions workspace | Provides information regarding HTTP sessions created by each Web application. HTTP sessions are used to maintain a user session between multiple invocations and stores user specific data. This workspace also displays the total, average, maximum, and minimum sizes of the user data stored for each Web application. |
| EJB Containers workspace | Provides an overview of the performance of different types of Enterprise JavaBeans (EJBs) deployed in the application server. |
| EJB workspace | Displays the performance data for each EJB deployed in the application. Based on the Bean type, data is available in the appropriate columns. |
| Container Transactions workspace | Provides performance data regarding Java Transaction API (JTA) transactions the EJBs are involved in. |
| Container Object Pools workspace | Provides information about the behavior of stateless and entity bean pools. |
| DB Connection Pools workspace | Displays the usage data regarding Database connection pools such as the number of connections available, checked out, threads waiting for a connection, and so on. This workspace is useful in understanding the pool usage and identifying potential bottlenecks in application performance when a thread is waiting or getting timed out for a connection from the pool. |
| J2C Connection Pools workspace | Displays the usage data for connection pools set up for Java™ Connector Architecture (JCA) based resource adapters. The WebSphere Application Server creates connection pools for the resource adapters that are deployed (some of the adapters are provided by application server installation itself) and this workspace is useful in monitoring performance bottlenecks in terms of wait times to obtain a connection. |
| Pool Analysis workspace | Provides information about the usage of several types of pools associated with each application server, including Web container pools, ORB pools, J2C connection pools, and database connection pools. This workspace helps you detect resource constraints and potential performance congestion. |
| Thread Pools workspace | Displays the usage data for thread pools. The data can be used to determine whether the pool is configured correctly to service user requests. For example, the Web Container thread pool is used to execute servlet requests from the users and if the number of threads in the pool is small compared to the number of incoming requests, there will be a delay in servicing them leading to slower response times. |
| Thread Pool Trend workspace | Displays trend information about thread pool size and usage. |
| Cache Analysis: Dynamic Cache Cache Analysis workspace | Provides the usage data of the various dynamic caches that
have been configured in the system. In WebSphere Application Server Version
6 or later, multiple caches can be configured. Also provides the usage data for each dynamic cache template. The templates are unique IDs that are specified in the cachespec.xml files to identify different URLs that must be cached. |
| Workload Management workspace | Workload management distributes the user requests made through
the Object Request Broker (ORB) to different servers in a cluster.
This usually means that this feature will come into play only when
remote EJB calls are made over the ORB. This feature is applicable
only in the ND (Network Deployment) environment when clusters are
set up, so that the workload can be distributed when remote EJB calls
are invoked over the ORB. If the ORB is not used, this feature is not exercised and there will be no data in this workspace. If a server has both Servlets and EJBs on the same server instance, the calls are usually configured to be local calls instead of remote calls for better performance, and hence the ORB will not come into play. The server side data provides information about the number of requests received on the server side (for example, the EJB container receiving the user requests). |
| Scheduler workspace | Scheduler service runs periodic tasks. Schedulers are persistent transactional timer services that run Enterprise JavaBean methods or send Java Message Service messages using any J2EE server application. |
| Web Services workspace | Provides performance data on the Web Services hosted by the
application server instance. Web Services provides data regarding
number of web services loaded, number of requests delivered, size
of the requests, and so on. Also includes information on the Web services gateway, which is used to map an existing service - either an inbound or an outbound service - to a new web service that appears to be provided by the gateway. The gateway acts as a proxy: your gateway service users need not know whether the underlying service is being provided internally or externally. The gateway provides you with a single point of control, access and validation of web service requests, and allows you to control which web services are available to different groups of web service users. |
| Workplace Mail workspace | Provides aggregated statistics of the usage information about the incoming message traffic. |
| Messages Queues workspace | Provides information about message delivery state, including ready retry, unprocessed, and dead. |
| Messaging workspace | The messaging engines defined in various Service Integration Buses in this application server. Provides performance numbers on each messaging engine such as number of messages published. This workspace displays data only when a Service Integration Bus (SIB) is available in the system. |
| Client Communications workspace | Provides communication details with distinct client processes that are currently network-connected to the Service Integration Buses of this application server. The data deals with messages sent and received from and to the various client processes. |
| Messaging Engine Communications workspace | Provides information about other application servers that are hosting messaging engines and are network-connected to this application server. The Service Integration Bus is used to send and receive messages from these messaging engines. |
| Destinations workspace | Provides performance data and counters for the destinations of a selected messaging engine. This includes the numbers of available and unavailable messages, numbers of messages produced and consumed, aggregated wait times and so on. |
| WMQ Client Link Communications workspace | Provides information regarding communication to WebSphere MQ JMS clients that are connected to this application server. |
| WMQ Link Communications workspace | Provides information regarding communication to WebSphere MQ JMS queue managers that are connected to this application server. |
| Alarm Manager workspace | Provides aggregated information about the alarms for each work manager. This includes the number of alarms created, fired, canceled and so on. |
| DCS Stacks workspace | Provides aggregated information about each DCS stack within the entire WebSphere Application Server domain, including multiple nodes and servers. This includes the incoming and outgoing message size, the number of incoming and outgoing messages, congestion events, and message buffer re-allocations. |
| Durable Subscriptions workspace | Provides statistic counters for the durable subscriptions of a selected topic. |
| High Availability Manager workspace | Provides aggregated information about high availability managers. |
| IMAP/POP workspace | Provides aggregated statistics of the usage information about the IMAP service and the POP3 service connectivity, especially for the performance-related connectivity. |
| Service Components workspace | Provides overview performance of the key service components. WebSphere servers feature their own service components, and each of these components has its own set of event points that can be monitored. |
| Service Component Elements workspace | Lists performance metrics for all the service components and their elements. Service components contain one or more elements, which are sets of different steps processed in each service component. In turn, each element has its own set of event natures, which are key points that are reached when processing a service component element. |
The availability of data in these workspaces depends on whether PMI is turned on in the WebSphere Application Server, and on the level of PMI data collection that is turned on.
PMI might be enabled at None, Basic, Extended, All, Custom levels. Each of these levels turns on a set of data counters. The higher level is inclusive of all the counters turned on at the lower level. As you go higher, the cost overhead for the collection increases. The "low" level incurs the least cost (usually obtains data that displays various "counts") and the "maximum" level incurs the highest cost (this, for example, returns time average data that requires extra memory and processing).
There is an additional "custom" level where individual data attributes can be turned on in addition to the data collection levels based on cost.
Hence, to view the Resource data in the portal workspaces, you will need to complete the following steps:
- Have the appropriate resources defined. For example, JDBC providers and Datasources defined with connection pools to see data in DB Connection pools, Schedulers defined to see any data in the Schedulers workspace, and so on.
- Turn on the PMI monitoring in the WebSphere Application Server (either through the admin console, wsadmin, or custom scripts).
- Select the appropriate level of PMI data collection based on the data that needs to be monitored.
For a list of the PMI categories and attributes that the WebSphere agent workspaces and tables use, see Mapping Cloud Pak System Software Monitoring Portal console workspace columns to WebSphere PMI attribute levels (WebSphere versions 7, 8, and 8.5).
Request Data
The Request data traces user transactions and measures the time to perform various activities (such as database access and JMS operations). Depending on the Request Data workspace, this information is grouped into various categories. For example, in the Request Analysis workspace the data is grouped by the request name (either the URL for servlets/JSPs or the EJB name) and in DataSources workspaces by the name of the Datasource.
The Monitoring Agent for WebSphere Application Server tracks the user transactions by modifying application and WebSphere system Java classes by adding method entry and exit "hooks". These hooks are callback methods into the Monitoring Data Collector for WebSphere module that will track a user transaction as it travels through the classes in various modules (servlets, EJBs, Datasources, JMS, and so on). The application and WebSphere Application Server system Java classes are modified when they are loaded by WebSphere classloaders. During this load time, based on the type of class - Servlets, EJB, Datasource, JMS, JCA, JNDI, and so on - different callback methods are injected into the class using "Byte Code Modification" (BCM).
| Workspace | Monitoring level | Description |
|---|---|---|
| Request Analysis workspace | Level 1 | This workspace displays the number of user requests that were tracked during the sampling interval and their average response times. The requests are grouped by URLs for servlet calls and EJB names for direct EJB calls. |
| Request Baseline workspace | Level 1 | This workspace displays aggregated information about the request baseline. In the baselining process, the WebSphere agent collects statistical information about a request completion times and uses this information to assign "fair" and "bad" thresholds for requests. |
| Request Analysis workspace | Level 2 | In addition to Level 1 data, the workspace also shows the breakdown of the response times into application, SQL Query, SQL Update, JCA, JMS, JNDI, SCA, web services processing times. This breakdown can be displayed in milliseconds and as a percentage of average response time. |
| Selected Request - Datasources workspace Selected Request - JMS Queues workspace |
Level 2 | To access this workspace, clicking the link icon on the row header of each row in the Request Analysis workspace. This workspace provides more details on the resources accessed by the selected user transaction - including the resource names, the average response time, and the longest response time during the sampling interval. |
| Data sources workspace | Level 2 | This workspace displays the time spent on various Data source operations. The operations include obtaining a connection, and executing a query or update. The workspace also shows the number of connections used during the sampling interval. The data in this workspace supplements the DB Connection Pools workspace by identifying the reason for long connection checkout times. |
| JMS Summary workspace | Level 2 | This workspace provides count and time measurements for JMS operations: send, receive, browse, and publish. It displays data on a queue and topic basis. The data provides a breakdown of the JMS delay shown in the Request Analysis workspace. |
The Monitoring Level determines the amount of collected request data. At Level 2 the WebSphere agent collects more detailed data and incurs additional overhead in terms of memory and processing time. The default is Level 1. To change the value, use the Start_Request_Monitoring: Begin reporting request data Take Action command.
Another parameter that determines the amount of data collected is the Request Data Sampling Rate. It specifies the percentage of user requests that the WebSphere agent tracks and monitors. When the load on the system is very large, tracking every single user request becomes very expensive. Therefore, the Agent uses a sampling approach to monitor the performance of the applications. The default is 2% (i.e. only 2 out of every 100 user requests are tracked).
However, the load on your system might be very low (for example, in a test system you might apply a 10 user load). In this case, you might see no data in the Request Data workspaces with the default setting of 2%, as the probability of tracking one of the 10 requests will be very low. Hence, you need to increase the sampling rate in systems with low load. Use the Set_Request_Sampling_Rate: Set the sampling rate for request data Take Action command to do this.
Log File Data
WebSphere agent reads the application server log files and analyzes them to display data in several workspaces.
| Workspace | Log file | Description |
|---|---|---|
| Garbage Collection Analysis workspace | Native error file specified in application server configuration. The file name is set when configuring the Data Collector (the default is gc.log). | WebSphere agent
scans the verbose garbage collection (GC) log to provide information
about how the GC is performing. This workspace can identify application
performance issues when the GC consumes a lot of time. Such issues
usually happen when the heap size parameters in the JVM are not set
correctly. You can stop and start the collection of GC data using
the Start_GC_Monitoring and Stop_GC_Monitoring Take Action commands. Important: the Take Action command does not turn on and off
the verbose GC flag in WebSphere JVM.
This flag is required for collection of GC data. You might need to
enable or disable it using the application server admin console.
|
| Allocation Failures workspace | Same as Garbage Collection Analysis | Provides information about heap allocation failures that cause the JVM to invoke garbage collection. |
| JVM Stack Trend workspace | Same as Garbage Collection Analysis | Provides trend data regarding JVM CPU usage, JVM garbage collection, and JVM heap usage. Displays Operating System data for CPU usage. |
| Log Analysis workspace | SystemOut.log | The WebSphere agent scans the application server SystemOut.log file and retrieves the messages for display in this workspace. Use the workspace to be informed about the health of the application server and also to create situations when certain message IDs are encountered. |
| Log Analysis workspace | Data Collector message events | Displays the Data Collector diagnostic messages, informing whether the Data Collector has initialized and is working correctly. |
Application server process data from the operating system
WebSphere agent obtains the system CPU usage for the application server process by making operating system calls. This data is displayed in the Application Server Summary and Business Process Manager Summary workspaces in the CPU Used (ms) and CPU Used (%) columns. The JVM Stack Trend workspace also uses this data.
The OS Stack workspace displays detailed operating system performance information for the application server.
WebSphere summary and aggregated information
The following workspaces contain summary or aggregated information for WebSphere Application Server.
| Workspace | Description |
|---|---|
| Application Server Summary and Business Process Manager Summary workspaces | This workspace displays overall statistic information for each application server that is monitored by the WebSphere agent . The information includes history of heap usage, response times, request rates, and % CPU used. |
| WebSphere Agent workspace | This workspace displays events occurring within the WebSphere agent and all the application servers on the host. It also displays status information about the agent and the online/offline status of the monitored servers. |
| EJB Tier Analysis workspace | The workspace displays information about the health of a WebSphere application based on response time and completion rates for EJB requests. It also displays ORB thread pool information and PMI information related to EJB requests. |
| Backend Tier Analysis workspace | The workspace displays information about the health of a WebSphere application based on response time for Backend Tier requests. It also displays information about backend tier resources, including Data Sources, JMS resources, JDBC and JCA Pools, and JVM statistics. |
| Web Tier Analysis workspace | The workspace displays information about the health of a WebSphere application based on response time and completion rates for Web Tier requests. It also displays WebContainer thread pool information and PMI information related to HTTP sessions. |
z/OS region workspaces
In a z/OS® environment, most workspace tables report data at both a region and server instance level. The advantage is that you can view metrics collected at both levels and switch between server instance lever and region level.
Data Display Problems
If you do not see data in the portal workspaces, use the following checklist to verify the settings.
| Workspace | Checkpoint | Comments |
|---|---|---|
| All Workspaces | The application server is running and the monitoring WebSphere agent is connected to it | Check that the Status column in the Application Server Summary and Business Process Manager Summary workspaces shows Connected. |
| All Workspaces | There is user load applied on the application server | The Request data workspaces display data only when user load is applied. The Resource data workspaces still display data rows even if load is not applied, but the columns will either be empty or display zeros. The data rows in this case are created from the application server configuration information about the resource type. |
| Request Data Workspaces | Request Data collection is turned on | Verify by checking the Request Data Monitoring Level column in the Application Server Summary workspace is not Disabled. If it is Disabled, start the monitoring by using the Start_Request_Monitoring: Begin reporting request data Take Action command. |
| Request Data Workspaces | Sampling rate is high enough | The default sampling rate is 2%. If the load on the application server is low, the WebSphere agent might fail to track user transactions. Verify the sampling rate in the Request Data Sampling Rate(%) column in the Application Server Summary and Business Process Manager Summary workspaces. Set the rate higher by using the Set_Request_Sampling_Rate: Set the sampling rate for request data Take Action command. |
| Data sources workspace | Request Data Monitoring Level is set to Level 2 | The Request Data Monitoring Level column in Application Server Summary and Business Process Manager Summary workspaces workspace displays the value. To change the monitoring level, use the Start_Request_Monitoring: Begin reporting request data Take Action command. |
| Data sources workspace | Data sources are configured in the application server; user requests are accessing these Data sources | To check the presence of Data sources, use the application server admin console. You need to know application logic to verify whether the user requests are accessing the Data sources. Check DB Connection Pools workspace for activity with the Data sources. |
| JMS Summary workspace | JMS resources (queues, topics) are configured and applications are accessing these resources | To check the existence of JMS resources, use the admin console. You need to know application logic to verify whether these resources are used. |
| Resource Data Workspaces | Feature is available | Verify the application server feature is being used. For example, Dynamic cache data will be available only when the cache feature has been set up. All "Platform Messaging" workspaces will be available only when the Service Integration Bus has been configured and used. |
| Resource Data Workspaces | PMI is enabled in the application server | Verify PMI is enabled through the administration console of WebSphere Administrative Console. Make sure PMI is enabled for the appropriate modules (for example, WebApplications, ServletSessions, EJBContainer, and so on). To do this, check the Runtime tab of the Performance Monitoring Infrastructure page in the WebSphere Administrative Console. |
| Resource Data Workspaces | Data not available in certain columns | Verify PMI level is high enough to capture data for these columns.
Check the Instrumentation Level column in the
workspace that you need to view. Tip: Change the PMI level
through the application server admin console. The contextual help
on the data columns in portal workspace specifies the PMI level at
which data will be available.
|
| Resource Data Workspaces | Resource Data Monitoring is turned on | The Resource Data Monitoring column in Application Server Summary and Business Process Manager Summary workspaces must say Enabled. If it is set to Disabled, turn it on using theStart_Resource_Monitoring: Begin reporting PMI data Take Action command. |
| Garbage Collection Analysis workspace | Verbose GC flag is turned on in WAS | Verify the Verbose GC box is checked in the Application Server->Process Definition->Java Virtual Machine page of the admin console. |
| Garbage Collection Analysis workspace | Garbage Collection Monitoring is enabled | The Garbage Collection Monitoring column
inApplication Server Summary and Business Process Manager Summary workspaces must
be Enabled. If it set to Disabled,
use the Start_GC_Monitoring: Begin reporting garbage-collection data Take Action command to start
it. Important: the Take Action command will not enable
the verbose GC flag in the application server. You have to do it separately.
|