TADDM Services & Support

Matching: rules

Programmatically Creating a Business Application Template

comaxb Tags: rules mql apptemplate api ‎ 2,641 Views

I am working with a customer this week and they have over 150 Business Applications that they have created. Unfortunately, they created them by using MQL rules similar to the following:

select * from SqlServer where exists (lower(modules.name) contains 'crs')

The problem with this rule is that it adds in the PCRS, UCRS, DCRS databases which are used for Production, UAT and Development respectively.

Now, you might think to yourself, gee, this is pretty easy, just re-create a business application specific to production and put in the full database name like 'PCRS' and then do the same thing for Development, UAT and Test. Did I mention that they have 150 apps? Assuming we need to create 3 apps for each of the base apps, that is 450 apps! Cutting and pasting the rules is onerous. The TADDM UI doesn't currently have any way to copy a business application template, so I opened an Enhancement Request (http://www.ibm.com/developerworks/rfe/).

Since I needed the capability this week, I built a jython script that makes a copy of an AppTemplate along with its corresponding MQLRules. This script can be downloaded here: jython script to copy an AppTemplate

Copy the script to $COLLATION_HOME/support/bin and invoke it as follows:

$ ./cpba.jy administrator collation "TADDM" "TADDM - Production"

It will login to TADDM as "administrator" with the password "collation" and make a copy of the TADDM AppTemplate to the new name "TADDM - Production".

Initially, I created the app and just set the new name to whatever was passed in on the command line (something like "TADDM - production")

What I found was that the resulting business app would not show up in grouping composer nor in the list of business apps. In addition, I set the MQL rules to the same values as were in the source Business App. This caused problems because the name of the MQL rule was the same in both apps meaning that the GUID for the MQL rules was also the same, hence changes to a rule in one business app would cause the rule in the other business app to change.

What I found was that I had to do the following:
1) Create a business app and set its name to the name passed in on the command line (like "TADDM - production") and get the GUID returned
2) The name of the AppTemplate had to be set to BA_GUID:BA_NAME
(something like "AA0A20EE5BBD336481279CA664FB380A:TADDM - Production")
3) The names of the rules had to be prefixed with the BA_GUID as well - but with no colon
(something like "AA0A20EE5BBD336481279CA664FB380Adatabase")

The javadoc or class documentation showing these attributes should be updated to indicate the required format and I have opened a Documentation PMR on that.

Building a Business Application in TADDM - Part 1

comaxb Tags: rules composer mql grouping ‎ 1 Comment ‎ 5,507 Views

Introduction

This document details the procedure used to map a complex Business Application in TADDM. One thing that will become clear as you read through this paper is that the role of the Subject Matter Expert for the Business Application is critical and cannot be eliminated. If no SME is available to tell you that the Business Application uses a DB2 database, then the road to Mapping the Business Application will be exceedingly long, if not impossible. Without the SME, you will have to rely on “connection” information from known components of the App to find the unknown components.

The Customer Portal Business Application

I am going to map the Customer Portal business application. It is a real application from a real customer, however, I have made changes where necessary to protect the identity of the customer. The Business Application is composed of the following components:

F5 Load Balancers
Oracle 11g on Solaris
DB2 on Z mainframe
DB2 on AIX
WebSphere MQ
WebSphere Application Server 6.1
IBM WebSeal
IBM Tivoli Access Manager (TAM)
Tealeaf Customer Experience Software
Microsoft IIS
Microsoft SQL Server
Team Site
Live Site
Apache Web Server
Apache Tomcat

Discover the Components

The first step in building a business application is to ensure that all components are properly discovered by TADDM. No work is required for those components that are supported by TADDM out of the box, however, Custom Server Templates (and often Extensions) and Computer System Templates will need to be created in order to completely map the Application. Of the components listed above, the majority are supported out of the box. The following applications will require custom server templates (and possibly extensions).

IBM WebSeal
- CST Criteria: Program name contains webseald
IBM TAM

CST Criteria: Windows Service Name contains “Access Manager”
Since Tivoli Access Manager is a WebSphere Application, if you have WebSphere discovery enabled and you have the access list populated with the username, password and certificates.

TeaLeaf
Team Site
Live Site
Open Deploy

Options

There are three methods that can be used to cause a component to become a part of a Business Application.

Manual - should be avoided as structural changes to the application will require effort to keep the business application up to date. Sometimes using a manual component is the only practical way to place a component in a business application.
Application Descriptors - formerly the solution of choice requiring small XML files to be deployed to the various target computer systems.
MQL Rules - statements similar to SQL which are executed against the TADDM server to select components to be added to the business application. You should strive to use MQL rules whenever possible.

The Business Application builder is not restricted to using only one of the methods above. Indeed, the best solution usually involves a combination of one or more techniques.
Mapping the Business Application

As mentioned above, it is best to use MQL Rules to add components to Business Applications and we will use this method when possible.

Login to the TADDM Data management portal and go to the "Grouping Composer" located in the Discovery Drawer.
Click to create a New Group and name it "Customer Portal - Production"
Leave the type set to Business Application, then click "Next"

On the next screen, you have the option to create one or more MQL rules. Keep it Simple! Use a separate rule for each Component type that you want added - its even ok to use multiple rules for the same component type.

Adding DB2 to the Business Application

The first step in writing a rule to add a DB2Instance to the Business Application is to look at the details panel for the DB2 that you know to be part of the App. A good place to look is on the Modules tab for an AppServer or on the Databases tab for a database. The following diagram illustrates the technique. Below is the databases tab for a DB2 that I have in my TADDM server.

If I was mapping my TADDM installation as a business application, I would be fairly sure that this database server was a part of it. Why? Because one of the database names contains the string "TADDM".

How do I compose my MQL?

The best way to validate that your MQL is returning the objects that you want (and not returning the ones you don't) is to test your MQL from the command line before you save it to the rule.

For example, I knew from the general tab that the type was “DB2 Instance”.

I could have a quick look in the model documentation (available in the file CDMWebsite.zip in $COLLATION_HOME/dist/sdk/doc/model on the installed TADDM server), or I could just guess that the type of the object was DB2Instance. To test to make sure I had the right object type, I used:

$ ./api.sh -u administrator -p collation find “select * from DB2Instance”

…. Of course, this promptly failed with a “no such object” error.

I know that our model usually uses camel case (Capital first letter, lower remainder of the word, so I tried again:

$ ./api.sh -u administrator -p collation find “select * from Db2Instance”

The screen fills with xml data indicating that my command worked well. Next, we need to narrow our search to find only the instances in which we are interested. To do this it's helpful to look at the object displayed as XML:

$ ./api.sh -u administrator -p collation find “select * from Db2Instance” > /tmp/db2.xml

Load the file /tmp/db2.xml into an editor and look for the TADDM721 database. You will see that there are modules (some stuff removed from output for brevity):

<modules array="1" guid="8DDF70524E0830DA84DA46B7D2460D7A" lastModified="1319209020785"
parent="14E62C6FAE3735F0800B6E1B58B71083" xsi:type="coll:com.collation.platform.model.topology.app.db.db2.Db2Module">
<name>TADDM721:SYSIBMADM</name>
<fileName>TADDM721:SYSIBMADM</fileName>
<displayName>TADDM721:SYSIBMADM</displayName>
</modules>

There are also databases:

<databases array="1" guid="333B613E41713D4BAB4FAF903519F515" lastModified="1319209020914"
parent="14E62C6FAE3735F0800B6E1B58B71083" xsi:type="coll:com.collation.platform.model.topology.app.db.db2.Db2Database">
<displayName>TADDM721</displayName>
<name>TADDM721</name>
</databases >

I can now try the rule to select only the DB2 Instances that have a database name that contains the string "TADDM".

select * from Db2Instance where exists (databases.displayName contains 'TADDM')

Enter the above query string into the rule and press "save":

The rule name can be anything you like, but it's best to choose something descriptive. The Functional group name is a simple string that is only used when comparing one Business Application with another. For example, a single business application could have two Apache Servers - one which serves as the customer interface and one which is used by the Application Administrators to adjust the settings of the application. These two Apache servers would have different Functional Group Names.

Click next and you are taken to a screen where you can add manual components to the business application:

For now, we won't add any manual components, so press Next, then press “Finish”. After a short period of time, you will see the New Business Application appear in the UI. You can then View the Topology Graph for this Business Application.

Further Restricting the Rule

You “may” notice something irregular about the graph. It's possible that you will see DB2s in there that don't belong to the Production Customer Portal. It's likely that the UAT or Development Customer Portal DB2 would also be named “TADDM721”. One of the best ways to solve this problem is to further restrict the MQL rule by specifying that the hostname match some criteria. This will only work reliably for customers who have a strong hostname convention in place. At this particular company, hostnames will be of the form: CIXYYYYZNNNN

Where the “Z” character is D, S, or P for “Development”, “Staging”, or “Production” respectively.

If MQL supported matching on regular expressions, the additional syntax would be slight:

select * from Db2Instance where exists (databases.displayName contains 'TADDM.*') and host.fqdn matches 'CI.....P.*'

Unfortunately, MQL does not currently support regular expression pattern matching (I've opened RFE 14280 to have this implemented), so our MQL rule has the potential to miss some DB2 Instances. According to the host naming standard, “YYYY = Server Type - Alpha acronym for Server Type, i.e. IIS, SQL, etc.”. Assuming that there is an Alpha acronym for DB2 and further assuming that it is “DB2”, then our MQL becomes:

select * from Db2Instance where exists (databases.displayName contains 'TADDM.*') and host.fqdn contains 'DB2P'

Oracle 11g and MS SQL Server

The procedure for adding Oracle 11g and MS SQL to to the Business Application is nearly identical to the procedure outlined above for DB2. The process is:

View the details panel for the component and find something that uniquely identifies that component as belonging to the Business Application (usually a database with a specific name)
Find out the type of the object by either looking at the common data model documentation (CDMWebsite.zip) or by making a logical guess based on the Object Type field shown on the General tab of the details panel
Test the select statement from the command line (./api.sh)
Place the rule in the Grouping Composer

What about Custom Servers?

As was mentioned earlier in this document, TADDM does not discover IBM WebSeal out of the box. In order to discover it, we created a Custom Server Template named “IBM WebSeal” with criteria Program Name contains webseald. When we run discovery, all WebSeal servers should be discovered correctly. How do we pick the right webseal(s) to add to the Business Application? We can base it on the content of the configuration file. Assuming the webseald configuration is in the file /opt/pdweb/etc/webseald-default.conf, the Custom Server Template “Config File” tab would look like this:

Ideally, this file would be collected and parsed with a custom server extension and the “junctions” would be collected and created as modules. I have opened RFE 13343 requesting IBM to add support for discovery of WebSeal out of the box.

The following MQL restricts the WebSeals that will be added to the business application based on the contents of the config file:

select * from AppServer where objectType == 'IBM WebSeal' and exists (configContents.content.content contains 'Portal WebServer')

In part 2, we will address three other items which are important to building business applications:

Custom server templates for z/OS started tasks
Computer System SNMP templates for unrecognized network equipment
A sample custom server extension to populate modules for software that is not discovered out-of-the-box

Feed for Blog Entries

Blogs