- How monitoring data is collected and viewed in PureApplication
- The topology for DataPower appliances and the ITM monitoring solution
- 1. Deploy and configure the DataPower appliance
- 2. Set up the System Monitoring for DataPower shared service
- 3. View workspaces in the System Monitoring portal
- Downloadable resources
- Related topics
Monitor DataPower virtual appliances from PureApplication System
Set up Tivoli monitoring services to collect and view system health, resource consumption, alerts, and more
The IBM® WebSphere® DataPower® Appliances (referred to as DataPower) is a hardware platform designed to simplify, secure, and accelerate XML, Web services, and Enterprise Service Bus deployments. In IBM PureApplication® System, a DataPower appliance is presented as a virtual appliance running in a virtual machine. The virtual machine provides the same functions as a real DataPower appliance for other workloads running on PureApplication System.
The PureApplication monitoring solution for DataPower leverages the IBM Tivoli® Monitoring (referred to as ITM) and IBM Tivoli® Composite Application Manager (referred to as ITCAM) product families. The solution integrates these products with the PureApplication pattern framework to provide the monitoring capabilities of collecting, storing, displaying, aggregating, and analyzing data from the DataPower virtual appliance. This data includes status, resource usage, logs, and events.
To set up the system that is described in this tutorial, you must install:
- IBM PureApplication System, PureApplication Software, or PureApplication Service V2.2 or greater
- Foundation pattern type V2.1.6 or greater
- System Monitoring pattern type V1.0.7 or greater
- DataPower Monitoring pattern type V1.0.1. This pattern type can be obtained either by downloading the individual DataPower Monitoring pattern type from the PureApplication Fix Center or by upgrading PureApplication Default Data to V2.2 where the DataPower Monitoring pattern type is entitled.
How monitoring data is collected and viewed in PureApplication
The DataPower appliance itself is equipped with a set of monitoring utilities for collecting general system health, resources and services consumption, alerts and situations, and services action records. The appliance shows the collected data to external systems in various ways for historical storage, performance analysis, and problem diagnosis. In PureApplication System, the ITCAM agent for the DataPower appliance (referred to as ITCAM agent) captures and intercepts the collected data as follows:
- Invokes the XML management interface that is implemented in the DataPower appliance to request device and services status and configuration through standard SOAP interfaces.
- Uses Simple Network Management Protocol (SNMP) over User Datagram Protocol (UDP) to poll an SNMP agent running on the DataPower appliance for device and application metrics. Listens for notification alerts from the agent in response to particular events that are happening on the appliance. The agent responds to inbound polling requests and sends alerts in response to preconfigured events.
- Reads the log files created and maintained by the local log server. The local log server receives remote messages from the DataPower appliance that runs a log agent to route logs back to the server by using the SysLog protocol.
The ITCAM agent communicates with ITM servers and transports the performance data, events, and logs collected from the DataPower appliance to ITM servers in near real-time. The servers include:
- The Tivoli Enterprise Portal Server, which creates the workspaces and displays attributes and situations predefined by the ITCAM agent.
- The Tivoli Enterprise Servers, which aggregate the collected data for further analysis.
- The Tivoli Data Warehouse, which stores the historical data.
In PureApplication System, the Tivoli Enterprise Portal is customized to be a PureApplication System monitoring portal. This portal presents DataPower monitoring data as shown in Figure 1.
Figure 1. Workspaces for DataPower appliances in the System Monitoring portal
The workspaces for DataPower appliances display the following information:
- The HTTP connection, including the statistics of HTTP connections to DataPower appliances and transactions on the HTTP connections.
- Events, including the subscribed SNMP traps for the events and alerts generated in DataPower appliances.
- Network, including the status of Ethernet interfaces and TCP ports used in DataPower appliances, and the received and transmitted throughputs on the interfaces.
- System logs, including the latency logs and system logs subscribed for various levels of log events generated in DataPower appliances.
- System information, including the status of DataPower domains, objects, and services.
- Usage, including the usage of CPU and system memory, the memory status of domains and services, the status of file systems, the system load, and the resource usage summary.
The topology for DataPower appliances and the ITM monitoring solution
In PureApplication System, a DataPower appliance is deployed as a classic virtual system pattern. Multiple virtual system instances can be launched for multiple DataPower appliances. All ITM servers are run together as the System Monitoring shared service. This shared service monitors not only DataPower patterns but also other workload patterns such as WebSphere application servers, WebSphere message queues, HTTP servers, IBM Integration Bus, and DB2®. Each of these workloads has specific ITM and ITCAM agents to run in patterns or out of patterns and each communicates with the System Monitoring shared service. The ITCAM agent for DataPower appliance is deployed to a separate virtual machine from the DataPower appliance and also runs as a shared service: the System Monitoring for DataPower shared service. This service collects data from different DataPower appliances. Figure 2 shows the topology of DataPower appliances and the ITM monitoring solution in PureApplication System.
Figure 2. Topology of DataPower appliances and ITM monitoring solution in PureApplication System
The System Monitoring for DataPower shared service provides:
- Lifecycle management of the ITCAM agent. The service launches a virtual machine and installs the ITCAM agent on the virtual machine when it is deployed. The service then configures the ITCAM agent to connect to one of ITM servers in the System Monitoring shared service. It reconfigures the ITCAM agent to connect to a new ITM server when the old server is removed or broken. It uninstalls the ITCAM agent when the agent is deleted.
- Operations for configuring the ITCAM agent to communicate with DataPower appliances. You use these operations to specify DataPower information such as the XML management interface port and the listening port for SNMP traps. These ports enable data transmission from the designated DataPower appliances to the ITCAM agent.
PureApplication System uses cloud groups or environment profiles to isolate workload patterns that run for different environments, such as for production, test, or staging. To guarantee the connectivity between an ITCAM agent and DataPower appliances, and between an ITCAM agent and ITCAM servers, deploy the System Monitoring for DataPower shared service in the same cloud group or environment profile as the System Monitoring shared service. All DataPower appliances to be monitored must have both System Monitoring and System Monitoring for DataPower running in their own cloud group or environment profile. However, if you configured multiple external management environment profiles to be reachable by each other on a network, you can also distribute DataPower appliances, System Monitoring, and System Monitoring for DataPower into different environment profiles.
Deploy and configure the DataPower appliance
You must enable and configure monitoring capabilities for each DataPower appliance to make the monitoring data accessible to the ITCAM agent. You can use the command-line interface (CLI) or the web management console to configure the DataPower appliance. The steps are to deploy the appliance, enable SOAP interfaces, configure SNMP settings, and set up the SysLog service.
Step 1. Deploy the DataPower appliance
In PureApplication, the DataPower virtual appliance is included with the hypervisor edition image and is deployed as a virtual system pattern. Before the pattern is launched, you must specify a set of properties to initialize the appliance with the default settings. As shown in Figure 3, the left box shows the DataPower appliance in the PureApplication view, and the right box is the deployment pane that shows the required properties. The properties Enable XML Management Interface and XML Management Interface port are used to enable SOAP interfaces for monitoring after the appliance is started. Enable SOAP interfaces describes these properties in detail. These two properties can be reset at runtime by using the DataPower appliance management console if the Enable Web Management Service property is set to True.
Figure 3. Deploying the DataPower virtual appliance pattern
Step 2. Enable SOAP interfaces
You must enable SOAP interfaces so that the ITCAM agent can send SOAP requests to the DataPower appliance. The appliance wraps the collected data into the SOAP responses.
- Open the XML Management Interface page by selecting
Objects > Device Management
or search "XML Management Interface" in the console. Figure 4 shows
the SOAP settings in the web management console of the DataPower
Figure 4. Configuring SOAP settings for the DataPower appliance
- Complete the required settings as follows:
- Administrative State. Set this field to enabled.
- Port Number. Enter the port number on which the ITCAM agent listens for notification reports. The port number is 5550 by default. Use different port numbers for different DataPower appliances if the appliances are expected to connect by using the same System Monitoring for DataPower shared service.
- Enabled services. Select SOAP management URI.
Step 3. Configure SNMP settings
The SNMP traps that are generated in a DataPower appliance must be configured and propagated to the ITCAM agents.
- Open the SNMP settings window. Select Administration
> Access or search "SNMP Setting" in the console.
Figure 5 shows the window for SNMP settings in the web management
Figure 5. Configuring SNMP settings for a DataPower appliance
- Complete the required settings as follows:
- Administrative State. Set this field to Enabled.
- Local Port. Enter the UDP port number that is monitored by the SNMP agent or engine for incoming SNMP requests. The reserved port number for the SNMP agent is 161.
- SNMPv3 Users. Enter an SNMP v3 user ID.
- SNMPv3 Security level. Select the security level.
- Open the Trap and Notification Targets page to specify trap and notification targets. For the Remote Host Address field, enter the host name or IP address of the computer that hosts the ITCAM agent. For the Remote Port field, enter the number of the port that the ITCAM agent listens on. Use different port numbers for different DataPower appliances if the appliances are expected to be connected by using the same System Monitoring for DataPower shared service.
- Go to the Trap Event Subscriptions page to subscribe the SNMP traps to be sent to the ITCAM agent.
Step 4. Set up the SysLog service
Enable the SysLog service to deliver the log messages from different DataPower appliance components to the ITCAM agent.
- Find Manage Log Targets by selecting Administration
> Miscellaneous or by searching "Manage Log
Targets" in the console. Figure 6 shows the window for the SysLog
settings in the web management console.
Figure 6. Configuring the SysLog service of a DataPower appliance
- Complete the required settings as follows:
- Administrative State. Select enabled.
- Target Type. Select syslog.
- Syslog Facility. Select the appropriate syslog facility to be included in messages sent to the ITCAM agent.
- Remote Host. Enter the host name or IP address that hosts the ITCAM agent that is running in the System Monitoring for DataPower shared service. The host name or IP address can be found in the PureApplication shared service instance page.
- Remote Port. Enter a port number. The port number that is reserved for syslog is 514, which is the only port the System Monitoring for DataPower supports.
- Go to the Event Subscriptions page to add the logs to be sent to the ITCAM agent.
Set up the System Monitoring for DataPower shared service
After you configure the DataPower appliance, you can use the System Monitoring for DataPower shared service to engage the ITCAM agent for collecting data from DataPower appliances. The following steps show you how to install the pattern type and operate the shared service.
Step 1. Install the DataPower Monitoring pattern type
The System Monitoring for DataPower shared service is implemented in the DataPower Monitoring pattern type, which is entitled in PureApplication System V2.2.
- If you are not using PureApplication System V2.2, download the pattern
type package from the PureApplication Fix Center, and install it by
selecting PureApplication Catalog >
Pattern Types, as shown in Figure 7.
Figure 7. Importing the DataPower Monitoring pattern type
- Install and enable Foundation pattern type V22.214.171.124 or later. The DataPower Monitoring pattern type depends on the Foundation pattern type.
- Enable the DataPower Monitoring pattern type. Click Enable to change
the status of the pattern type to "Available", as shown in Figure 8.
Figure 8. Enabling the DataPower Monitoring pattern type
Step 2. Deploy system monitoring for DataPower
The System Monitoring shared service is required for provisioning ITM servers for the ITCAM agent to connect to. Therefore, before you deploy the System Monitoring for DataPower shared service, ensure that the System Monitoring shared service is already deployed.
- When the DataPower Monitoring pattern type is successfully enabled, select Patterns > Shared Service to view the System Monitoring for DataPower shared service. Click Deploy to open the deployment window.
- Specify the environment profile, cloud group, IP group, operation
system family, image, and SSH key (Figure 9). Choose
Linuxfor the Operating system family field. The System Monitoring for DataPower shared service supports only Linux in PureApplication System. Deploying the shared service launches a virtual machine and installs the ITCAM agent on the machine.
Figure 9. Deploying the System Monitoring for DataPower shared service
Step 3. Configure the ITCAM agent for the DataPower appliance
The System Monitoring shared service provides a group of operations for configuring the ITCAM agent for DataPower appliances. After the System Monitoring for DataPower shared service instance enters RUNNING status, the operations are ready to use.
- Open the management console through the shared service instance page
and select ITM-DataPower.ITCAMBN to find the
operations under the Operations tab. There are five
operations as shown in Figure 10.
Figure 10. Operations of the System Monitoring for DataPower shared service
- Use the Add Instance operation to add one DataPower appliance into the
monitored object. The operation creates a new instance of the ITCAM
agent and binds it to a specific DataPower appliance. The instance
must be configured with DataPower appliance settings to collect
monitoring data. These settings are specified in the properties shown
in Figure 11.
Figure 11. Properties of the Add Instance operation
- The values that you input must be aligned with the settings that you
used in enabling SOAP interfaces and configuring SNMP settings on the
DataPower appliance side. There is no additional setting for log
monitoring except for the option of enabling syslog monitoring,
because the System Monitoring for DataPower shared service supports
only the SysLog protocol and port 151 is used by default. The
following table explains all the properties.
Table 1. DataPower appliance properties
Properties Description Instance Name The name that is given to the DataPower appliance that you want to monitor. Ensure that the name is unique if you are monitoring more than one DataPower appliance. SNMP Event Port The port number of the SNMP event. Different DataPower appliance instances must use different port numbers for SNMP events. SNMP Event User The user name for SNMP event. SNMP Event Security Level The event security level of SNMP. SNMP Authentication Protocol The authentication protocol used in SNMP. SNMP Authentication Secret The password for authentication protocol. SNMP Privacy Protocol The privacy protocol used in SNMP. SNMP Privacy Secret The password for privacy protocol. XML Management Interface Port The port that the XML Management Interface is using. User ID The ID of the DataPower administrative user. Password The password for the administrative user. Device Host The IP address of the DataPower appliance. Enable syslog monitoring Check this box to enable syslog monitoring.
- Use the Query Instance operation to query the settings of an existing
ITCAM agent instance that is bound to a DataPower appliance. The
operation returns the values that were input through the Add Instance
or Update Instance operation. The output is displayed in the Return
Value field of Operation Execution Results at the
bottom of the page (Figure 12).
Figure 12. Example of a Query Instance result
View workspaces in the System Monitoring portal
When you successfully create an ITCAM agent instance for a DataPower appliance, the instance is also configured to connect to the ITM servers of the System Monitoring shared service. The instance transmits the health status, event, and log messages of the appliance to the ITM servers. In the System Monitoring portal, you can view the workspaces for the data collected from the appliance. The links to launch the portal are on the System Monitoring for DataPower shared service instance page as shown in Figure 13.
Figure 13. Links in the System Monitoring portal
When multiple DataPower appliances are added into the System Monitoring shared service, they are displayed in the System Monitoring portal. As shown in Figure 14, two appliances bn1 and bn2 are listed under the node of the DataPower Monitoring Agent. Each appliance has its own workspace for the resource usage, health status, events, and log reports.
Figure 14. Example of multiple DataPower appliances in the System Monitoring portal
As a cloud platform, PureApplication System provides runtime resources and management for DataPower appliances. It also provides the monitoring capabilities of collecting, gathering, and presenting performance data, events, and logs from the appliances. This tutorial described how to enable monitoring for DataPower appliances in PureApplication System. You learned how to set up and configure the shared services required for appliances to communicate with the ITCAM agent and ITM servers. You also learned how to view resource usage, health status, and other statistics for appliances and other patterns that run in PureApplication System.
- Configuring the ITCAM Agent for WebSphere DataPower Appliance
- Webcast replay: Monitoring DataPower with ITCAM
- Three-part series on Monitoring WebSphere DataPower
- IBM Knowledge Center: WebSphere DataPower SOA Appliances
- IBM Knowledge Center: PureApplication System