Verify the CloudBurst 2.1 Tivoli software stack

The advantages an appliance brings with it are often achieved by complex tasks; many times this complexity is hidden by the interface to the appliance, giving the user a limited view of the entire configuration and integration points. But a user may need to verify or re-verify the software stack when the environment changes (restoring backup images in a disaster recovery scenario) or when making modifications to hardware configurations (like when you add new blades) or software configurations (like when you add new networks with VLAN tagging). In this article, the author provides a quick guide to verifying the IBM® CloudBurst 2.1 Tivoli® software stack.

Rossella De Gaetano (rossella.degaetano@it.ibm.com), Test Lead, IBM

Rossella De Gaetano is the test leader for IBM CloudBurst 2.1 and IBM Service Delivery Manager 7.2.1. She is responsible for the quality assurance of the Tivoli software stack included both in IBM CloudBurst and IBM Service Delivery Manager. Previously she was the development leader for Tivoli Asset Management for distributed and the License Metric Tool.



26 August 2011

Also available in Chinese Japanese

The usage of interconnected virtual images as a way of deploying software stacks is becoming more prevalent. A typical approach is to pre-configure the software stack to minimize the effort to implement the solution. While this approach works well for the initial implementation, care must be taken to re-verify configurations after significant environmental changes such as:

  • Restoring a set of backup images as part of a disaster recovery scenario.
  • Modifying the hardware configuration (like adding new blades).
  • Modifying the software configuration (like adding networks, external hypervisors).

This article describes a series of steps customers should follow to ensure that their virtual image environment is properly verified after significant environmental changes. IBM® CloudBurst 2.1 pre-integrated software solution is used as an example of the steps necessary to ensure success. The same methodology can be used to verify other pre-configured software stacks.

Introducing testing CloudBurst 2.1

The CloudBurst 2.1 software stack is made up of pre-installed and pre-configured virtual images that contain the IBM products to exploit the critical functionalities needed for cloud computing:

  • Tivoli® Service Automation Manager (TSAM): Offers the core functionality for a cloud computing environment. It allows the user to request, create, delete, modify, and manage the virtual images in the cloud.
  • IBM Tivoli Monitoring (ITM): Keeps availability and performance management of virtual images and the provisioned virtual images under control.
  • Tivoli Usage and Accounting Manager (TUAM): Keeps track of the resource consumption for billing functions.

The practical implementation of virtual image environments like CloudBurst 2.1 requires a re-verification of the software environment after significant changes in the environment. This article outlines a methodical series of steps to ensure the operation and health of the entire environment. This involves an approach that moves from the specific to the general:

  1. Check the health status of every single virtual image from the networking point of view.
  2. Check the software stacks inside each virtual image to ensure they are up and operating correctly.
  3. In relation to the software, check the integration points between the virtual images.

Completing these steps will ensure the health and operation of your CloudBurst environment.

Please note that for sake of simplicity, the tests described do not include the optional dual-node high availability feature.


Step 1: Checking the network configuration

First we'll walk through the issues to check the network configuration.

Default network configuration

The default network configuration of CloudBurst 2.1 management images is shown in Figure 1.

Figure 1. CloudBurst default network configuration
CloudBurst default network configuration

In Figure 1, four virtual images are shown. Three of the virtual images (icb-tsam, icb-tuam, icb-itm) are equipped with two virtual Network Interface Cards (vNICs) and one of them (icb-nfs) has three vNICs. The vNICs for all four virtual images are defined in the hypervisor.

Other network interfaces (the ones without vNICs) are created in one of two ways, either using VLAN tagging or with Tivoli System Automation for Multiplatforms (TSA).

VLAN tagging, also known as IEEE 802.1Q, is an IEEE 802.1 work group networking standard for the sharing of a physical Ethernet network link by multiple independent logical networks. It defines the meaning of a virtual LAN (VLAN) with respect to the conceptual model that defines bridging at the MAC layer and to the IEEE 802.1D Spanning Tree Protocol (which allows nodes on different VLANs to communicate with one another through a network switch with OSI layer 3 network layer capabilities or through a router.

If the customer is exploiting the multiple networks feature of Tivoli Service Automation Manager, the networking picture will have more NICs created with VLAN tagging and TSA.

Verifying network interfaces are up

The first step here is to check that all these network interfaces are up and properly configured:

Figure 2. icb-tivsam network configuration
icb-tivsam network configuration

Looking at the hypervisor, you should observe for each virtual image the following settings:

  • The first network adapter is connected to VM Network.
  • The second network adapter is connected to the Management Network.
  • The third network adapter (for icb-nfs only) is connected to the Customer Network.

If you login to each of the management virtual images, the ifconfig command shows all the defined network adapters. The following table shows all the IP address defined by default in CloudBurst 2.1:

Table 1. CloudBurst 2.1 default IP addresses
Virtual imageicb-tivsamicb-tivsam-ha*icb-nfsicb-nfs-ha*icb-itmicb-tuam
eth010.90.0.110.90.0.310.90.0.510.90.0.610.90.0.710.90.0.8
eth1192.168.88.1192.168.88.3192.168.88.5192.168.88.6192.168.88.7192.168.88.8
eth2Customer definedCustomer defined
eth0.10010.100.0.110.100.0.310.100.0.510.100.0.610.100.0.7
eth0:010.90.0.210.90.0.210.90.0.410.90.0.4
eth1:0192.168.88.2192.168.88.2192.168.88.4192.168.88.4
eth2:0Customer defined
eth0.100:010.100.0.210.100.0.210.100.0.410.100.0.4

*Note that the icb-tivamha and icb-nfs-ha virtual images exist only in case dual-node high availability is exploited.

How to do this: A successful ping using all the defined interfaces will demonstrate the network configuration is correct.

At a first glance, this test may look very basic and unnecessary, but consider that many of the problems in deploying virtual images (especially with IBM Tivoli Monitoring agent) and URL forwarding issues are often related to an incorrect network configuration.


Step 2: Checking the software stack

The power of the CloudBurst solution comes from the combined power of the single products included. With this power comes the complexity of its infrastructure. In this section, you'll see a simple way to ensure that the middleware and the applications are running properly.

All of the needed software is configured to automatically start up at boot. So check to ensure that all the software is up and running. You will need to go to each virtual image mentioned in the following subsections and perform the indicated identified checks.

Checks for the icb-itm virtual image

  • Check that DB2® is started. For DB2, it is enough for the db2inst1 user to launch db2start. It should return a message saying the database manager is already active.
  • Confirm that these six installed IBM Tivoli Monitoring components are running:
    • Tivoli Enterprise Monitoring Server
    • Linux® operating system agent
    • warehouse proxy agent
    • summarization and pruning agent
    • Tivoli Enterprise Portal Server
    • Eclipse Help Server

How to do this: For the ITM components, the output of /opt/IBM/ITM/cinfo -r shows them in the running state (they need to be started and stopped as virtuser).

Checks for the icb-nfs virtual image

  • Check that on icb-nfs, the HTTP server, the mail server, the network file system server, and the Samba server are running. Even if you are not exploiting dual-node high availability, all services are managed by TSA.

How to do this: To check that everything is running properly, look at the output of lssam -V and check that all resources are online, as shown in Figure 3.

Figure 3. Output of lssam -V for icb-nfs
Output of lssam -V for icb-nfs

In Figure 3, the resources include:

  • The resource samp-ihs-server represents the HTTP server.
  • The resource samp-nfsserver represents the NFS server.
  • The resource samp-samba-server represents the Samba server.
  • The resource samp-postfix-server represents the mail server.
  • The resource samp-mgmt-virtual-ip represents the service IP address on eth1.
  • The resource samp-virtual-ip represents the service IP address on eth0.
  • The resource samp-prvs-eth0.100 represents the service IP address on eth0.100.

Note the output shown in Figure 3 does not contain the service IP address for eth2; it had not been added when I took this screenshot.

Checks for the icb-tivsam virtual image

  • Check that on icb-tivsam, the DB2 instances that belongs to ctginst1 and dasusr1, the ldap, WebSphere, Tivoli Provisioning Manager, and Tivoli Service Automation Manager are up and running.

As for icb-nfs, even if not exploiting dual-node high availability, all services are managed by TSA.

How to do this: To check that everything is running properly, look at the output of lssam -V You need to check that all resources are online, as shown in Figure 4.

Figure 4. Output of lssam -V for icb-tivsam
Output of lssam -V for icb-tivsam

In Figure 4, the resources include:

  • The resource db2_ctginst1_0-rs represents the DB2 instance used by Tivoli Service Automation Manager.
  • The resource db2_idsccmdb_0-rs represents the DB2 instance used by Tivoli Directory Server (ldap).
  • The resource ids-rs represents the ldap.
  • The resource tpm_tio-rs represents Tivoli Provisioning Manager.
  • The resource was_MXServer-rs represents the MXServer.
  • The resource was_dmgr-rs represents the WebSphere Deployment Manager.
  • The resource was_nodeagent-rs represents the WebSphere node agent.
  • The resource was_server1-rs represents the server1 application server.
  • The resource was_webserver-rs represents WebSphere Application Server.
  • The resource samp-mgmt-virtual-ip represents the service IP address on eth1.
  • The resource db2ip-rs represents the service IP address on eth0.
  • The resource samp-prvs-eth0.100 represents the service IP address on eth0.100.

Checks for the icb-tuam virtual image

  • Check the icb-tuam virtual image to ensure that DB2 and the application server have been started.

How to do this: For DB2, it is enough if the db2inst1 user launches db2start. If the database manager is running, db2start returns a message saying the database manager is already active.

How to do this: As root from the application server, launch /opt/IBM/tuam/ewas/bin/serverStatus.sh server1 -username virtuser -password <virtuser's password>. If the server is up and running the command returns a message saying that the server is started.

Details on how to start and stop the software stack are found in CloudBurst 2.1 infocenter, in the Resources section.

Final software started check

Finally, login to the products' user interfaces:

  • ITM: https://192.168.88.7:1920 (default user sysadmin).
  • TSAM: https://192.168.88.2/SimpleSRM (default user PMRDPCAUSR).
  • TPM: https://192.168.88.7/maximo (default user maxadmin).
  • TUAM: http://192.168.88.8:11052 (default user virtuser).

Logging in successfully to the respective user interfaces (UI) demonstrates the individual software has been started correctly.


Step 3: Checking the atomic software functionalities

Once all the checks in Step 2 have been successfully verified, you can start having a look at the specific product functionalities. Let's say you are effectively moving from "unit test" to "functional test."

So, let's start with the easiest and quickest ones first.

Functional test for Network File System (icb-nfs)

For this virtual image, the functional test is meant to ensure that the network file system server and the Samba server are properly working. A failure here compromises the capability to perform deployments with ITM agent.

How do to this: The easiest way to check that these two services are working is to try to use them. From the virtual image server, try to connect to the exported file system by opening up Windows Explorer and entering \\192.168.88.4\repository and logging in with Administrator /<Administrator's password>. The results are shown in Figure 5.

Figure 5. Testing the Samba server
Testing the Samba server

If you plan to provision Windows® virtual images, you may want to do an analogous check from one of the provisioned virtual images before trying to deploy a virtual image together with the ITM agent. Of course you need to use the proper icb-nfs address on the management network specified in the Tivoli Service Automation Manager administrative user interface (Figure 6).

Figure 6. Cloud pool network settings
Cloud pool network settings

It is easier to check the network file system server. Since you do not need a provisioned virtual image, you can simply use one of the management virtual images. For this example, let's use icb-itm. Create a temporary directory there (/mytestdir) and then try to mount the exported file system: mount 10.100.0.4:/repository /mytestdir. As for the Samba server, remember to do the tests with the icb-nfs IP address that corresponds to the management network defined in Tivoli Service Automation Manager administrative user interface.

Functional test for IBM Tivoli Monitoring Server (icb-itm)

For icb-itm, a straightforward test is to check that the ITM agents running on the management virtual images are showing up in the user interface:

Figure 7. ITM user interface
ITM user interface

In this way you check that both the monitoring products are started and that the connection to the database is working.

Functional test for Tivoli Usage and Accounting Server (icb-tuam)

For icb-tuam, it is also enough to check that you can login to the user interface and navigate it (Figure 8).

Figure 8. TUAM user interface
TUAM user interface

Functional test for Tivoli Service Automation Manager (icb-tivsam)

For Tivoli Service Automation Manager, the functional tests are not as simple. Suppose that you already created a team; the main test you need to do is to create a project. This is the only way to check that the entire configuration you defined in the administrative user interface is correct and consistent.

Tivoli Service Automation Manager 7.2.1 offers improved error checking, especially for the network configuration. There is still nothing better than attempting a provisioning. I suggest you first try a simple provisioning (that is, no ITM agent or additional software added), and then move onto more complex scenarios (the next section has more information on this). That way you can limit the variables in case you have to perform problem determination.

How to do this: Before provisioning, check in the Tivoli Service Automation Manager administrative UI to see all the provisioning blades among the Provisioning Computers. Moreover, you may want to check that the configuration of the cloud pool you are going to use for provisioning is correct:

  1. Navigate to Go To > Service Automation > Cloud Pool Administration.
  2. Select your cloud pool.
  3. Click the Cloud Pool Network Settings tab and check to see that the pool is attached to the proper networks and that one that is available by icb-tivsam has the Use as management Interface checkbox checked.

Next, you can verify that the corresponding resource pool is configured for the proper provisioning cluster: Go To > Administration > Provisioning > Resource Pools.

The last thing to check is that the networks attached to the cloud pool have the proper variables defined for blocked IP addresses and VLAN IDs: Go To > IT Infrastructure > Provisioning Inventory > Subnetworks.

Once this is complete, login to the TSAM Web 2.0 user interface and create a project. Be sure to select the right resource pool.

After the provisioning completes successfully, you should receive an email notification from the Tivoli Service Automation Manager application that contains the administrative password to access the virtual image. Connect to the image and check the network configuration is the one you expected.


Step 4: Checking the integration points

If all the previous steps worked, you are ready to verify the integration points:

  • URL redirection.
  • TSAM-ITM: Create a virtual image equipped with ITM agent.
  • TSAM-TUAM: Display invoice reports.

URL redirection

The URL redirection is the functionality that allows you to access TSAM administrative and Web 2.0 user interfaces and TUAM user interfaces using the icb-nfs IP address. So the straightforward test here is to try to open these UIs with the proper address:

  • TSAM administrative UI: https://icb-nfs/maximo.
  • TSAM Web 2.0 UI: https://icb-nfs/SimpleSRM.
  • TUAM UI: https://icb-nfs/ibm/console.

How to do this: According to the IP address you use to access icb-nfs, be sure the system from which you use the web browser can access the corresponding network. For example if you use the browser on the virtual image server, the redirection will work using https://icb-nfs/maximo or https://192.168.88.4/maximo, but it will not work if you use https://10.90.0.4/maximo because the virtual network server does not have access to that network.

TSAM and ITM integration

The master test to check TSAM-ITM integration is to create a project checking the check box Monitoring Agent to be Installed. Here you need to check that not only the service request ends up in a resolved state, but also that the Tivoli Monitoring UI shows the agent and the corresponding monitored data:

Figure 9. ITM user interface
ITM user interface

If you added additional networks with VLAN tagging, the ITM agent must be installed as additional software. The check box Monitoring Agent to be Installed works only for eth0.100.

TSAM and TUAM integration

To check the integration between TSAM and TUAM there needs to be a day's worth of data collected. You need to wait one day after a successful provisioning or de-provisioning before that data is available in the invoice reports.

How to do this: One day after the provisioning/de-provisioning, the following things must be checked:

  • Verify that on icb-tivsam, in /var/IBM/TSAM/metering, an RDP_*<date>.txt file has been created and its size is different from zero. Each day a file is created with the data corresponding to the previous day.
  • Verify on icb-tuam, in /opt/IBM/tuam/samples/logs/collectors, the file checked in the previous instruction is present (it gets moved there by a cron job).
  • In the TUAM UI, if you go to Tivoli Usage and Accounting Manager > Chargeback Maintenance > Load Tracking, you can see data imported for the specified day.
  • In the TSAM administrative UI, you can display the invoice reports: Go To > Administration > Report Administration.
Figure 10. TUAM total invoice report
TUAM total invoice report

In conclusion

The steps described in this article are the key ones that need to be executed in order to verify the health status of your CloudBurst software stack. These suggested tests cover the basic functionalities of each of the software products included in the solution and all the predefined functional integration points.

These tests should be the core checks to be done after applying changes to your CloudBurst environment; for example, after restoring a set of backup images in a disaster recovery scenario or after upgrading the software stack.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Cloud computing on developerWorks


  • Bluemix Developers Community

    Get samples, articles, product docs, and community resources to help build, deploy, and manage your cloud apps.

  • Cloud digest

    Complete cloud software, infrastructure, and platform knowledge.

  • DevOps Services

    Software development in the cloud. Register today to create a project.

  • Try SoftLayer Cloud

    Deploy public cloud instances in as few as 5 minutes. Try the SoftLayer public cloud instance for one month.

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Cloud computing, Tivoli
ArticleID=754177
ArticleTitle=Verify the CloudBurst 2.1 Tivoli software stack
publish-date=08262011