![Close [x]](http://dw1.s81c.com/i/c.gif){kind=small}
The usage of interconnected virtual images as a way of deploying software stacks is becoming more prevalent. A typical approach is to pre-configure the software stack to minimize the effort to implement the solution. While this approach works well for the initial implementation, care must be taken to re-verify configurations after significant environmental changes such as:
- Restoring a set of backup images as part of a disaster recovery scenario.
- Modifying the hardware configuration (like adding new blades).
- Modifying the software configuration (like adding networks, external hypervisors).
This article describes a series of steps customers should follow to ensure that their virtual image environment is properly verified after significant environmental changes. IBM® CloudBurst 2.1 pre-integrated software solution is used as an example of the steps necessary to ensure success. The same methodology can be used to verify other pre-configured software stacks.
Introducing testing CloudBurst 2.1
The CloudBurst 2.1 software stack is made up of pre-installed and pre-configured virtual images that contain the IBM products to exploit the critical functionalities needed for cloud computing:
- Tivoli® Service Automation Manager (TSAM): Offers the core functionality for a cloud computing environment. It allows the user to request, create, delete, modify, and manage the virtual images in the cloud.
- IBM Tivoli Monitoring (ITM): Keeps availability and performance management of virtual images and the provisioned virtual images under control.
- Tivoli Usage and Accounting Manager (TUAM): Keeps track of the resource consumption for billing functions.
The practical implementation of virtual image environments like CloudBurst 2.1 requires a re-verification of the software environment after significant changes in the environment. This article outlines a methodical series of steps to ensure the operation and health of the entire environment. This involves an approach that moves from the specific to the general:
- Check the health status of every single virtual image from the networking point of view.
- Check the software stacks inside each virtual image to ensure they are up and operating correctly.
- In relation to the software, check the integration points between the virtual images.
Completing these steps will ensure the health and operation of your CloudBurst environment.
Please note that for sake of simplicity, the tests described do not include the optional dual-node high availability feature.
Step 1: Checking the network configuration
First we'll walk through the issues to check the network configuration.
The default network configuration of CloudBurst 2.1 management images is shown in Figure 1.
Figure 1. CloudBurst default network configuration
In Figure 1, four virtual images are shown. Three of the virtual images (icb-tsam, icb-tuam, icb-itm) are equipped with two virtual Network Interface Cards (vNICs) and one of them (icb-nfs) has three vNICs. The vNICs for all four virtual images are defined in the hypervisor.
Other network interfaces (the ones without vNICs) are created in one of two ways, either using VLAN tagging or with Tivoli System Automation for Multiplatforms (TSA).
VLAN tagging, also known as IEEE 802.1Q, is an IEEE 802.1 work group networking standard for the sharing of a physical Ethernet network link by multiple independent logical networks. It defines the meaning of a virtual LAN (VLAN) with respect to the conceptual model that defines bridging at the MAC layer and to the IEEE 802.1D Spanning Tree Protocol (which allows nodes on different VLANs to communicate with one another through a network switch with OSI layer 3 network layer capabilities or through a router.
If the customer is exploiting the multiple networks feature of Tivoli Service Automation Manager, the networking picture will have more NICs created with VLAN tagging and TSA.
Verifying network interfaces are up
The first step here is to check that all these network interfaces are up and properly configured:
Figure 2. icb-tivsam network configuration
Looking at the hypervisor, you should observe for each virtual image the following settings:
- The first network adapter is connected to VM Network.
- The second network adapter is connected to the Management Network.
- The third network adapter (for icb-nfs only) is connected to the Customer Network.
If you login to each of the management virtual images, the ifconfig command shows all the defined network adapters. The following table shows all the IP address defined by default in CloudBurst 2.1:
Table 1. CloudBurst 2.1 default IP addresses
| Virtual image | icb-tivsam | icb-tivsam-ha* | icb-nfs | icb-nfs-ha* | icb-itm | icb-tuam |
|---|---|---|---|---|---|---|
| eth0 | 10.90.0.1 | 10.90.0.3 | 10.90.0.5 | 10.90.0.6 | 10.90.0.7 | 10.90.0.8 |
| eth1 | 192.168.88.1 | 192.168.88.3 | 192.168.88.5 | 192.168.88.6 | 192.168.88.7 | 192.168.88.8 |
| eth2 | Customer defined | Customer defined | ||||
| eth0.100 | 10.100.0.1 | 10.100.0.3 | 10.100.0.5 | 10.100.0.6 | 10.100.0.7 | |
| eth0:0 | 10.90.0.2 | 10.90.0.2 | 10.90.0.4 | 10.90.0.4 | ||
| eth1:0 | 192.168.88.2 | 192.168.88.2 | 192.168.88.4 | 192.168.88.4 | ||
| eth2:0 | Customer defined | |||||
| eth0.100:0 | 10.100.0.2 | 10.100.0.2 | 10.100.0.4 | 10.100.0.4 |
*Note that the icb-tivamha and icb-nfs-ha virtual images exist only in case dual-node high availability is exploited.
How to do this: A successful ping using all the defined interfaces will demonstrate the network configuration is correct.
At a first glance, this test may look very basic and unnecessary, but consider that many of the problems in deploying virtual images (especially with IBM Tivoli Monitoring agent) and URL forwarding issues are often related to an incorrect network configuration.
Step 2: Checking the software stack
The power of the CloudBurst solution comes from the combined power of the single products included. With this power comes the complexity of its infrastructure. In this section, you'll see a simple way to ensure that the middleware and the applications are running properly.
All of the needed software is configured to automatically start up at boot. So check to ensure that all the software is up and running. You will need to go to each virtual image mentioned in the following subsections and perform the indicated identified checks.
Checks for the icb-itm virtual image
- Check that DB2® is started. For DB2, it is enough for the db2inst1 user to launch
db2start. It should return a message saying the database manager is already active. - Confirm that these six installed IBM Tivoli Monitoring components are running:
- Tivoli Enterprise Monitoring Server
- Linux® operating system agent
- warehouse proxy agent
- summarization and pruning agent
- Tivoli Enterprise Portal Server
- Eclipse Help Server
How to do this: For the ITM components, the output of /opt/IBM/ITM/cinfo -r shows them in the running state (they need to be started and stopped as virtuser).
Checks for the icb-nfs virtual image
- Check that on icb-nfs, the HTTP server, the mail server, the network file system server, and the Samba server are running. Even if you are not exploiting dual-node high availability, all services are managed by TSA.
How to do this: To check that everything is running properly, look at the output of lssam -V and check that all resources are online, as shown in Figure 3.
Figure 3. Output of lssam -V for icb-nfs
In Figure 3, the resources include:
- The resource
samp-ihs-serverrepresents the HTTP server. - The resource
samp-nfsserverrepresents the NFS server. - The resource
samp-samba-serverrepresents the Samba server. - The resource
samp-postfix-serverrepresents the mail server. - The resource
samp-mgmt-virtual-iprepresents the service IP address on eth1. - The resource
samp-virtual-iprepresents the service IP address on eth0. - The resource
samp-prvs-eth0.100represents the service IP address on eth0.100.
Note the output shown in Figure 3 does not contain the service IP address for eth2; it had not been added when I took this screenshot.
Checks for the icb-tivsam virtual image
- Check that on icb-tivsam, the DB2 instances that belongs to ctginst1 and dasusr1, the ldap, WebSphere, Tivoli Provisioning Manager, and Tivoli Service Automation Manager are up and running.
As for icb-nfs, even if not exploiting dual-node high availability, all services are managed by TSA.
How to do this: To check that everything is running properly, look at the output of lssam -V You need to check that all resources are online, as shown in Figure 4.
Figure 4. Output of lssam -V for icb-tivsam
In Figure 4, the resources include:
- The resource
db2_ctginst1_0-rsrepresents the DB2 instance used by Tivoli Service Automation Manager. - The resource
db2_idsccmdb_0-rsrepresents the DB2 instance used by Tivoli Directory Server (ldap). - The resource
ids-rsrepresents the ldap. - The resource
tpm_tio-rsrepresents Tivoli Provisioning Manager. - The resource
was_MXServer-rsrepresents the MXServer. - The resource
was_dmgr-rsrepresents the WebSphere Deployment Manager. - The resource
was_nodeagent-rsrepresents the WebSphere node agent. - The resource
was_server1-rsrepresents the server1 application server. - The resource
was_webserver-rsrepresents WebSphere Application Server. - The resource
samp-mgmt-virtual-iprepresents the service IP address on eth1. - The resource
db2ip-rsrepresents the service IP address on eth0. - The resource
samp-prvs-eth0.100represents the service IP address on eth0.100.
Checks for the icb-tuam virtual image
- Check the icb-tuam virtual image to ensure that DB2 and the application server have been started.
How to do this: For DB2, it is enough if the db2inst1 user launches db2start. If the database manager is running, db2start returns a message saying the database manager is already active.
How to do this: As root from the application server, launch /opt/IBM/tuam/ewas/bin/serverStatus.sh server1 -username virtuser -password <virtuser's password>. If the server is up and running the command returns a message saying that the server is started.
Details on how to start and stop the software stack are found in CloudBurst 2.1 infocenter, in the Resources section.
Finally, login to the products' user interfaces:
- ITM: https://192.168.88.7:1920 (default user sysadmin).
- TSAM: https://192.168.88.2/SimpleSRM (default user PMRDPCAUSR).
- TPM: https://192.168.88.7/maximo (default user maxadmin).
- TUAM: http://192.168.88.8:11052 (default user virtuser).
Logging in successfully to the respective user interfaces (UI) demonstrates the individual software has been started correctly.
Step 3: Checking the atomic software functionalities
Once all the checks in Step 2 have been successfully verified, you can start having a look at the specific product functionalities. Let's say you are effectively moving from "unit test" to "functional test."
So, let's start with the easiest and quickest ones first.
Functional test for Network File System (icb-nfs)
For this virtual image, the functional test is meant to ensure that the network file system server and the Samba server are properly working. A failure here compromises the capability to perform deployments with ITM agent.
How do to this: The easiest way to check that these two services are working is to try to use them. From the virtual image server, try to connect to the exported file system by opening up Windows Explorer and entering \\192.168.88.4\repository and logging in with Administrator /<Administrator's password>. The results are shown in Figure 5.
Figure 5. Testing the Samba server
If you plan to provision Windows® virtual images, you may want to do an analogous check from one of the provisioned virtual images before trying to deploy a virtual image together with the ITM agent. Of course you need to use the proper icb-nfs address on the management network specified in the Tivoli Service Automation Manager administrative user interface (Figure 6).
Figure 6. Cloud pool network settings
It is easier to check the network file system server. Since you do not need a provisioned
virtual image, you can simply use one of the management virtual images. For this example,
let's use icb-itm. Create a temporary directory there (/mytestdir) and then try to mount
the exported file system: mount 10.100.0.4:/repository
/mytestdir. As for the Samba server, remember to do the tests with the icb-nfs IP
address that corresponds to the management network defined in Tivoli Service
Automation Manager administrative user interface.
Functional test for IBM Tivoli Monitoring Server (icb-itm)
For icb-itm, a straightforward test is to check that the ITM agents running on the management virtual images are showing up in the user interface:
Figure 7. ITM user interface
In this way you check that both the monitoring products are started and that the connection to the database is working.
Functional test for Tivoli Usage and Accounting Server (icb-tuam)
For icb-tuam, it is also enough to check that you can login to the user interface and navigate it (Figure 8).
Figure 8. TUAM user interface
Functional test for Tivoli Service Automation Manager (icb-tivsam)
For Tivoli Service Automation Manager, the functional tests are not as simple. Suppose that you already created a team; the main test you need to do is to create a project. This is the only way to check that the entire configuration you defined in the administrative user interface is correct and consistent.
Tivoli Service Automation Manager 7.2.1 offers improved error checking, especially for the network configuration. There is still nothing better than attempting a provisioning. I suggest you first try a simple provisioning (that is, no ITM agent or additional software added), and then move onto more complex scenarios (the next section has more information on this). That way you can limit the variables in case you have to perform problem determination.
How to do this: Before provisioning, check in the Tivoli Service Automation Manager administrative UI to see all the provisioning blades among the Provisioning Computers. Moreover, you may want to check that the configuration of the cloud pool you are going to use for provisioning is correct:
- Navigate to Go To > Service Automation > Cloud Pool Administration.
- Select your cloud pool.
- Click the Cloud Pool Network Settings tab and check to see that the pool is attached to the proper networks and that one that is available by icb-tivsam has the Use as management Interface checkbox checked.
Next, you can verify that the corresponding resource pool is configured for the proper provisioning cluster: Go To > Administration > Provisioning > Resource Pools.
The last thing to check is that the networks attached to the cloud pool have the proper variables defined for blocked IP addresses and VLAN IDs: Go To > IT Infrastructure > Provisioning Inventory > Subnetworks.
Once this is complete, login to the TSAM Web 2.0 user interface and create a project. Be sure to select the right resource pool.
After the provisioning completes successfully, you should receive an email notification from the Tivoli Service Automation Manager application that contains the administrative password to access the virtual image. Connect to the image and check the network configuration is the one you expected.
Step 4: Checking the integration points
If all the previous steps worked, you are ready to verify the integration points:
- URL redirection.
- TSAM-ITM: Create a virtual image equipped with ITM agent.
- TSAM-TUAM: Display invoice reports.
The URL redirection is the functionality that allows you to access TSAM administrative and Web 2.0 user interfaces and TUAM user interfaces using the icb-nfs IP address. So the straightforward test here is to try to open these UIs with the proper address:
- TSAM administrative UI: https://icb-nfs/maximo.
- TSAM Web 2.0 UI: https://icb-nfs/SimpleSRM.
- TUAM UI: https://icb-nfs/ibm/console.
How to do this: According to the IP address you use to access icb-nfs, be sure the system from which you use the web browser can access the corresponding network. For example if you use the browser on the virtual image server, the redirection will work using https://icb-nfs/maximo or https://192.168.88.4/maximo, but it will not work if you use https://10.90.0.4/maximo because the virtual network server does not have access to that network.
The master test to check TSAM-ITM integration is to create a project checking the check box Monitoring Agent to be Installed. Here you need to check that not only the service request ends up in a resolved state, but also that the Tivoli Monitoring UI shows the agent and the corresponding monitored data:
Figure 9. ITM user interface
If you added additional networks with VLAN tagging, the ITM agent must be installed as additional software. The check box Monitoring Agent to be Installed works only for eth0.100.
To check the integration between TSAM and TUAM there needs to be a day's worth of data collected. You need to wait one day after a successful provisioning or de-provisioning before that data is available in the invoice reports.
How to do this: One day after the provisioning/de-provisioning, the following things must be checked:
- Verify that on icb-tivsam, in /var/IBM/TSAM/metering, an RDP_*<date>.txt file has been created and its size is different from zero. Each day a file is created with the data corresponding to the previous day.
- Verify on icb-tuam, in /opt/IBM/tuam/samples/logs/collectors, the file checked in the previous instruction is present (it gets moved there by a cron job).
- In the TUAM UI, if you go to Tivoli Usage and Accounting Manager > Chargeback Maintenance > Load Tracking, you can see data imported for the specified day.
- In the TSAM administrative UI, you can display the invoice reports: Go To > Administration > Report Administration.
Figure 10. TUAM total invoice report
The steps described in this article are the key ones that need to be executed in order to verify the health status of your CloudBurst software stack. These suggested tests cover the basic functionalities of each of the software products included in the solution and all the predefined functional integration points.
These tests should be the core checks to be done after applying changes to your CloudBurst environment; for example, after restoring a set of backup images in a disaster recovery scenario or after upgrading the software stack.
Learn
-
For expanded information on the topics in this article:
- Start with the CloudBurst 2.1 infocenter.
- In case the tests described in this article failed, here's some troubleshooting results for CloudBurst and for TSAM.
- These CloudBurst tutorials should take you deeper into how to perform basic tasks like those in the article.
-
In the developerWorks cloud developer resources, discover and share knowledge and experience of application and services developers building their projects for cloud deployment.
-
More developerWorks resources that complement this article can be found on Tivoli at developerWorks and WebSphere at developerWorks.
-
The next steps: Find out how to access IBM SmartCloud Enterprise.
Get products and technologies
-
See the product images available on the IBM SmartCloud Enterprise.
Discuss
-
Join a cloud computing group on developerWorks.
-
Read all the great cloud blogs on developerWorks.
-
Join the developerWorks community, a professional network and unified set of community tools for connecting, sharing, and collaborating.
Rossella De Gaetano is the test leader for IBM CloudBurst 2.1 and IBM Service Delivery Manager 7.2.1. She is responsible for the quality assurance of the Tivoli software stack included both in IBM CloudBurst and IBM Service Delivery Manager. Previously she was the development leader for Tivoli Asset Management for distributed and the License Metric Tool.
Table of contents
Next steps from IBM
Learn more, join the cloud community, and get started with cloud computing tools from IBM
Tags
Use the slider bar to see more or fewer tags.
For articles in technology zones (such as Java technology, Linux, Open source, XML), Popular tags shows the top tags for all technology zones. For articles in product zones (such as Info Mgmt, Rational, WebSphere), Popular tags shows the top tags for just that product zone.
For articles in technology zones (such as Java technology, Linux, Open source, XML), My tags shows your tags for all technology zones. For articles in product zones (such as Info Mgmt, Rational, WebSphere), My tags shows your tags for just that product zone.
Popular article tags |
My article tagsSkip to tags list
Popular article tags |
My article tags
