If the 32-bit Tivoli Framework is updated to 6.3 Fixpack 7 on a system running the ISM, T3, T5, T6, TO, or TU agents, the agents crash on start up.
For the ISM agent - APAR IV93927
The fix for APAR IV93927 is available in 7.4.0.2-TIV-CAMIS-IF0001, see:
For the T3,T5,T6 agents - APAR IV93911
For the TO/TU agents - APAR IV93999
The symptom is that the agent log ends abruptly. Here is the what the log shows:
(58A3459F.0015-1:kbbssge.c,72,"BSS1_GetEnv") ICCRTE_DIR="/opt/IBM/ITM/li6243/gs"
(58A3459F.0016-1:kdebenc.c,364,"ssl_provider_constructor") TLS 1.0 protocol enabled
(58A3459F.0017-1:kdebenc.c,393,"ssl_provider_constructor") TLS 1.1 protocol enabled
(58A3459F.0018-1:kdebenc.c,416,"ssl_provider_constructor") TLS 1.2 protocol enabled
(58A3459F.0019-1:kbbssge.c,72,"BSS1_GetEnv") KDEBE_KEY_LABEL=GSK_KEY_LABEL="IBM_Tivoli_Monitoring_Certificate"
(58A3459F.001A-1:kbbssge.c,72,"BSS1_GetEnv") KDEBE_KEYRING_FILE=GSK_KEYRING_FILE="/opt/IBM/ITM/keyfiles/keyfile.kdb"
(58A3459F.001B-1:kbbssge.c,72,"BSS1_GetEnv") KDEBE_KEYRING_STASH=GSK_KEYRING_STASH="/opt/IBM/ITM/keyfiles/keyfile.sth"
<EOF>
The cinfo from an impacted system:
ax IBM Tivoli Monitoring Shared Libraries li6263 06.30.07.00
ax IBM Tivoli Monitoring Shared Libraries lx8263 06.30.07.00
ax IBM Tivoli Monitoring Shared Libraries lx8266 06.30.07.00
Here are two options for a work around.
Option 1 -
Install and run the affected agents on a different file system, like
'/opt/IBM/ITM2'
Option 2 -
Uninstall everything on the system, and install the ITCAM agents,
and then the 6.3 FP 6 OS agent, then upgrade the 32-bit frameworks to FP 6, with a command like this (command is all on one line):
/tmp/6.3.0-TIV-ITM_TMV-Agents-FP0006/install.sh -h /opt/IBM/ITM
-q -p /tmp/6.3.0-TIV-ITM_TMV-Agents-FP0006/unix/tflx8263.txt
/tmp/6.3.0-TIV-ITM_TMV-Agents-FP0006/install.sh -h /opt/IBM/ITM
-q -p /tmp/6.3.0-TIV-ITM_TMV-Agents-FP0006/unix/tfli6263.txt
Bruce Willman

Subscribe and follow us for all the latest information directly on your social feeds:
Modified on by harrietj 100000VRFW
|
How do I find this agent and what can it do for me?
After I install IBM Tivoli Composite Application Manager for Transactions V7.4.0.1 : Transaction Tracking - Linux, Multiplatform, English (CN6AZML ) I see the th agent in my cinfo. But there was no option to install it. I chose to install everything from the CN6AZML package and this is what shows up on the machine:

MQ Tracking (KTH agent) is an MQ exits-based data collector which has been available since ITCAM for Transactions V7.1. MQ Tracking uses MQ exits to extract the required data, which is then written to a local file or IPCSharedQueue. The data is then translated to Transaction Tracking API events and transferred to Transaction Collector. Check our documentation for more information about this hidden gem. https://www.ibm.com/support/knowledgecenter/en/SS5MD2_7.4.0.1/com.ibm.itcamt.doc/tt/dita/concept/kto_dom_mq_overview.html
Modified on by harrietj 100000VRFW
|
Create signed key store for Server
Step 1.
Set up the environment for GSKit CLI
export PATH=/opt/IBM/ITM/lx8266/gs/bin:$PATH
export LD_LIBRARY_PATH=/opt/IBM/ITM/lx8266/gs/lib64:$LD_LIBRARY_PATH
Step 2.
Create an empty key store. We will call this ‘SignedServerKey’.
The TEPS requires that the password be stashed in V1 format, so we must include the ‘v1stash’ parameter.
[root@n048028 keyfiles]# gsk8capicmd -keydb -create -empty -db /opt/IBM/ITM/keyfiles/SignedServerKey.kdb -pw ******** -expire 365 -v1stash -stash
Check the key store
[root@ nc049202 keyfiles]# gsk8capicmd -cert -list -db ./SignedServerKey.kdb -stashed
No certificates were found.
Step 3.
Create a new certificate request for the server, to be signed by the CA
There are two important things.
- The label should be "IBM_Tivoli_Monitoring_Certificate"
- The dn must match the server name
gsk8capicmd -certreq -create -db /opt/IBM/ITM/keyfiles/SignedServerKey.kdb -label "IBM_Tivoli_Monitoring_Certificate" -size 2048 -file ITMServerReq.arm -dn "CN=nc049202.tivlab.raleigh.ibm.com,OU=nc049202:LZ,O=IBM,L=RTP,ST=NC,C=US" -stashed
Step 4.
Use Certificate request to get the signed CA from certificate authority
There will be the signed certificate request, and at least one signer certificate.
Step 5.
Import signer certificate into key store
Use the label from the signer certificate. In this example, we use "Tivoli ITM Certificate Authority"
gsk8capicmd_64 -cert -add -file ./ITMRoot.arm -trust enable -label "Tivoli ITM Certificate Authority" -db /opt/IBM/ITM/keyfiles/SignedServerKey.kdb -stashed
Step 6.
Receive signed certificate into key store
gsk8capicmd_64 -cert -receive -file ./ITMServerReqSigned.arm -db /opt/IBM/ITM/keyfiles/SignedServerKey.kdb -stashed
Step 7.
Check key store
gsk8capicmd_64 -cert -details -label "Tivoli ITM Certificate Authority" -db SignedKey.kdb -stashed
gsk8capicmd_64 -cert -details -label IBM_Tivoli_Monitoring_Certificate -db SignedKey.kdb -stashed
gsk8capicmd_64 -cert -validate -label IBM_Tivoli_Monitoring_Certificate -db SignedServerKey.kdb -stashed
Create signer CA for test purposes
- Create ITM Root key store
gsk8capicmd_64 -keydb -create -db ITMRoot.kdb -pw trnsp3rf -expire 3650 -stash
- Create signer cert
For the dn, use the name of the TEMS server, and change the label as desired.
If you change the label, then substitute in all steps above.
gsk8capicmd_64 -cert -create -db ./ITMRoot.kdb -label "Tivoli ITM Certificate Authority" -size 2048 -dn "cn=nc049202.tivlab.raleigh.ibm.com,o=IBM,OU=ITM,L=RTP,ST=NC,C=US" -default_cert yes -expire 3695 -ca true -sigalg sha512
gsk8capicmd_64 -cert -list -db ITMRoot.kdb -stashed
gsk8capicmd_64 -cert -details -label "Tivoli ITM Certificate Authority" -db ITMRoot.kdb -stashed
- Extract signer cert
gsk8capicmd_64 -cert -extract -target ITMRoot.arm -db ./ITMRoot.kdb -label "Tivoli ITM Certificate Authority" -stashed
- Do steps above to create server key store and generate certificate request
- Sign request
gsk8capicmd_64 -cert -sign -db ITMRoot.kdb -label "Tivoli ITM Certificate Authority" -file ITMServerReq.arm -target ./ITMServerReqSigned.arm -stashed
- Continue steps to import CA and signed certificate
Useful links
Managing Certificates with GSKit
https://www.ibm.com/developerworks/library/se-gskit/
ITM Certificate configuration
https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Tivoli%20Monitoring/page/ITM%20Certificate%20Authentication%20Configuration%20Guide%20for%20ITM%206.2.2%20and%20newer%20releases
Modified on by harrietj 100000VRFW
|
Some web sites require strong encryption, for example to monitor HTTPS sites that use TLS1.2 ciphers. Once you have enabled strong encryption in RPT to run these scripts, you need to make corresponding changes to the T6 agent.
A common mistake is to forget to make these changes in the T6 agent's file system. Then, these scripts fail to play back. You will see a message like the following in the T6 agent's trace-robotic.log in the BWM logs directory:
Cannot support SSL_ECDHE_RSA_WITH_AES_256_CBC_SHA384 with currently installed providers
Cipher suites that use AES_256, for example, require installation of the JCE Unlimited Strength Jurisdiction Policy Files on the T6 agent hosts.
The simplest solution is to borrow a copy of the local_policy.jar and US_export_policy.jar files from your RPT workbench -- since the scripts are running correctly in that environment. Then, replace these files in the T6 agent file system. These are the jar files that allow the agent to support strong encryption.
Please note that the following solution assumes you are using the default T6 installation on Windows. The T6 pathnames are similar on Unix/Linux.
1) Log onto the RPT workbench host where you exported the RPT scripts to the AMC. Copy the following files from the RPT host to a scratch directory on a the T6 agent host.
C:\Program Files (x86)\IBM\SDP\jdk\jre\lib\security\local_policy.jar and US_export_policy.jar
2) Stop the T6 agent.
3) On the T6 agent host, copy the existing local_policy.jar and US_export_policy.jar files from the following directories to a backup directory under \tmp or \temp. Do not simply rename these files in their current directories. Do not try to create a backup directory for these jar files under the <ITM_HOME> directory.
Here, notice that we are replacing the T6 agent's jar files under the TMAITM6 folder.
<ITM_HOME>\TMAITM6\java60\jre\lib\security
<ITM_HOME>\TMAITM6\java70\jre\lib\security
<ITM_HOME>\TMAITM6\java80\jre\lib\security
4) Copy the local_policy.jar and US_export_policy.jar files that you saved in step 1 into these directories on the T6 agent host. Replace the existing files. These are the files you copied from the RPT workbench host.
<ITM_HOME>\TMAITM6\java60\jre\lib\security
<ITM_HOME>\TMAITM6\java70\jre\lib\security
<ITM_HOME>\TMAITM6\java80\jre\lib\security
5) Restart the T6 agent.
The following technotes are for reference only.
See Does the RRT agent support TLS 1.1/1.2 and 256-bit ciphers?
http://www.ibm.com/support/docview.wss?uid=swg21960133
See How to enable strong encryption > 128 bit
http://www.ibm.com/support/docview.wss?uid=swg21245273
See Cipher suites
https://www.ibm.com/support/knowledgecenter/en/SSYKE2_7.0.0/com.ibm.java.security.component.70.doc/security-component/jsse2Docs/ciphersuites.html
Modified on by harrietj 100000VRFW
|
The latest release of the ISM agent, 7.4.2.1 contains a fix for APAR IV47366 - ismconfig is unable to connect to ITM using secure ports.
7.4.0.2 7.4.0.2-TIV-CAMIS-IF0001 README:
https://www.ibm.com/support/docview.wss?uid=isg400003684
Here are the instructions to configure ismconfig.sh to use SSL:
To set ismconfig to use HTTPS, edit <ITM_HOME>/lx8266/cj/lib/ismconfig.props, and add the lines:
TepsPort : "3661"
TepsProtocol : "https"
If you have any issues, edit ismconfig.props and set logging to:
TraceParams : "Flow"
Then check /opt/IBM/ITM/logs/ismconfig.log for errors.
Modified on by harrietj 100000VRFW
|
Because HAR (HTTP ARchive' format ) files track the logging of the web browser's interaction with a web site, HAR files may be required to troubleshoot Synthetic (SN) agent issues such as the following:
-
Performance: slow page load, timeout when performing certain tasks
-
Page rendering: incorrect page format, missing information
This blog shows you how to generate a HAR file using the Firefox browser.
Getting Started
Important: If you cannot launch Firefox from the agent host, please capture the screenshot and HAR file from a host on the same network as the agent.
Before you visit the URL in Firefox again, please delete the browser cache first.
Launch Firefox. Click Ctrl+Shift+Delete in the Firefox client area. Choose "Everything" in "Time range to clear ". Click "Clear Now". Like this:

Capture a Screenshot and HAR File
1) Start by opening Firefox. Clear the browser cache.
2) In Firefox, enter the URL of the page within your application where you are experiencing trouble.
3) Click the Firefox menu (three horizontal parallel lines) at the top-right of your browser window. Then, select Developer > Network.

4) The Developer Network Tools will open as a docked panel at the side or bottom of Firefox. Click on the Network tab.
5) The recording will autostart once you start performing actions in the browser.
6) Click the Request Details button to display request details.

7) Refresh the web page you are on. The goal is to reproduce the problem you've been experiencing while Firefox is recording activity. In Firefox, you typically use the F5 key on your keyboard to refresh the current page.
Please try to notice if there is a request hang or lengthy pause in Firefox when you initially load the page. A lengthy response time may be partly due to the time it takes to open the page.
8) The Developer Network Panel needs to include the document load time and the time of the request. Like this:

9) Once you have reproduced the issue and you see that all of the actions have been generated in the Developer Network Panel (should just take a few seconds), right click under the "File" column. In the pop-up menu, click on "Save all as HAR". Another pop-up menu allows you to save the HAR file in your local file system. Save the HAR file somewhere convenient.

10) Take a screenshot of the Developer Network Panel.
11) You're done! Please attach the HAR file and the screenshot of the Developer Network Panel to your email, or upload these artifacts to the Salesforce case so IBM can assist you further.
Modified on by harrietj 100000VRFW
|
In the Transaction Reporter configuration, the Aggregation Agent List parameter specifies the Aggregation agents from which the Transaction Reporter collects aggregate data. If this field is blank, the reporter collects data from all available Aggregation agents. In addition to the TU (Transaction Collector) agent, the T5 (Web Response Time) agent can also be an Aggregation agent.
To specify Aggregation agents, enter the name of the agent, followed by :TU or :T5. Separate agent names with a space. For example:
host1:TU host2:TU host3:T5 host4:T5
See Transaction Reporter agent configuration parametershttps://www.ibm.com/support/knowledgecenter/en/SS5MD2_7.4.0.1/com.ibm.itcamt.doc/tt/dita/reference/kto_tools_datacoll_rep.html
IMPORTANT: Here, the use of the symbolic name ‘host’ is misleading. In the Aggregation Agent List, the references to the TU or T5 agents must be managed system names (MSN), not IP addresses or hostnames. Use the tacmd listSystems command to list the MSN for all TU and T5 agent hosts.
The tacmd listSystems command searches all of the managed systems connected directly to the hub TEMS and indirectly through remote TEMS servers.
Note that you have to log onto the hub TEMS and connect to the TEMS first.
a. Logon to the hub TEMS as root or as sysadmin.
b. Use the tacmd login command to log onto the hub TEMS service.
Windows:
<ITM_HOME>\BIN\tacmd login -u sysadmin -p <sysadmin-password> -s <full-hub-tems-server-name>
Unix/Linux:
<ITM_HOME>/bin/tacmd login -u sysadmin -p <sysadmin-password> -s <full-hub-tems-server-name>
The following messages should display:
Validating user...
KUIC00007I: User sysadmin logged into server on
https://<hub-tems-server-name>:<port>.
c. After you have established a connection to the hub TEMS service, enter the following tacmd listSystems command. Here, we show how to surface MSN’s for the TU agents. You can use a similar command to search for T5 agents. Substitute ‘t5’ for ‘tu’ in the tacmd listSystems command.
Windows:
ITM_HOME\BIN\tacmd listSystems -t tu
Unix/Linux:
ITM_HOME/bin/tacmd listSystems -t tu
Example:
# ./tacmd listSystems -t tu
Managed System Name Product Code Version Status
NC048034:TU TU 07.40.01.00 Y
In this example, The MSN is 'NC048034'. You would enter the following string in the Transaction Reporter's Aggregation Agent List:
NC048034:TU
Note: When the Aggregation Agent List is blank or commented out, the Transaction Reporter accepts data from all TU and T5 agents.
If a T5 and TU agent reside on the same host, insert a reference to the TU agent only in the Aggregation Agent List. Avoid using two aggregation agents from the same host.
If you have installed multiple Transaction Reporter agents in the same ITM system, you use the Aggregation Agent List to assign a subset of the available Aggregation agents to each Transaction Reporter agent. Ensure that there is no overlap of Aggregation agents in these assignments.
Configuring Transaction Reporter
On Windows, use the MTEMS (Manage Tivoli Enterprise Monitoring Services) GUI to reconfigure the TO agent. The parameter you need to edit is actually identified as ‘Aggregation Agent List’.

On Unix/Linux platforms, edit the <ITM_HOME>/config/to.ini file on the Transaction Reporter agent host. Here, the configuration variable is called ‘TU_LIST’. Remember to uncomment the TU_LIST variable.
Example:
# By default the Reporter contacts all available Aggregation Agents to collect aggregate data
# uncomment and set the following to specify a whitespace seperated list of Aggregation Agent names
# TU_LIST=Host1:TU Host2:TU Host3:T5 Host4:T5
Modified on by harrietj 100000VRFW
|
For efficiency, web servers often cache web pages for download to clients. This process is different than local browser caching or local Java caching.
Server caching can create issues for ISM monitors, because the content is static. In some instances, it may be possible to miss server irregularities that should be reported such as file not found (HTTP 404) errors.
The solution is to configure ISM monitors to read the uncached server content. You need to add the Pragma and Cache-Control headers to the Parameters tab for each URL monitor. You need both headers.
Name
|
Value
|
Type
|
Pragma
|
no-cache
|
HEAD
|
Cache-Control
|
no-cache
|
HEAD
|
Like this:

These headers enable the ISM URL monitors to bypass the server cache.
Note that you may need to also change the Version to 1.1 on the Advanced tab. The following screenshot shows that the version is set to 1.0.

Modified on by harrietj 100000VRFW
|
The HTTPS monitor reports an 'SSL Handshake Error' for a particular server.
Using the openssl command line from the ISM agent system, it is determined that the cipher used by the monitored server is 'ECDHE-RSA-AES256-GCM-SHA384'.
[root@nc050092 log]# openssl s_client -connect www.ibm.com:443
...
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES256-GCM-SHA384
...
Examine <ISM_HOME>/log/https.log, and if debug tracing is enabled, it will list the available ciphers:
Fri Jan 26 15:39:03 2018 566D4B70 Debug: Socket 9: SSL_new() succeeded SSL_CTX=0x56701F50
Fri Jan 26 15:39:03 2018 566D4B70 Debug: SSL_get_cipher_list Priority:0 Cipher:'ECDHE-RSA-AES256-GCM-SHA384'
Fri Jan 26 15:39:03 2018 566D4B70 Debug: SSL_get_cipher_list Priority:1 Cipher:'ECDHE-ECDSA-AES256-GCM-SHA384'
Fri Jan 26 15:39:03 2018 566D4B70 Debug: SSL_get_cipher_list Priority:2 Cipher:'ECDHE-RSA-AES256-SHA384'
Fri Jan 26 15:39:03 2018 566D4B70 Debug: SSL_get_cipher_list Priority:3 Cipher:'ECDHE-ECDSA-AES256-SHA384'
...
If the cipher required by the server is not listed, then the ISM agent might need to be updated to the latest version.
Here is a list of the 57 ciphers available in version 7.4.2.0 of the ITCAM ISM agent:
$(sslCipherSuiteList) -> "ECDHE-RSA-AES256-GCM-SHA384 ECDHE-ECDSA-AES256-GCM-SHA384 ECDHE-RSA-AES256-SHA384 ECDHE-ECDSA-AES256-SHA384 ECDHE-RSA-AES256-SHA ECDHE-ECDSA-AES256-SHA SRP-DSS-AES-256-CBC-SHA SRP-RSA-AES-256-CBC-SHA SRP-AES-256-CBC-SHA DHE-DSS-AES256-GCM-SHA384 DHE-RSA-AES256-GCM-SHA384 DHE-RSA-AES256-SHA256 DHE-DSS-AES256-SHA256 DHE-RSA-AES256-SHA DHE-DSS-AES256-SHA AECDH-AES256-SHA ADH-AES256-GCM-SHA384 ADH-AES256-SHA256 ADH-AES256-SHA ECDH-RSA-AES256-GCM-SHA384 ECDH-ECDSA-AES256-GCM-SHA384 ECDH-RSA-AES256-SHA384 ECDH-ECDSA-AES256-SHA384 ECDH-RSA-AES256-SHA ECDH-ECDSA-AES256-SHA AES256-GCM-SHA384 AES256-SHA256 AES256-SHA PSK-AES256-CBC-SHA ECDHE-RSA-AES128-GCM-SHA256 ECDHE-ECDSA-AES128-GCM-SHA256 ECDHE-RSA-AES128-SHA256 ECDHE-ECDSA-AES128-SHA256 ECDHE-RSA-AES128-SHA ECDHE-ECDSA-AES128-SHA SRP-DSS-AES-128-CBC-SHA SRP-RSA-AES-128-CBC-SHA SRP-AES-128-CBC-SHA DHE-DSS-AES128-GCM-SHA256 DHE-RSA-AES128-GCM-SHA256 DHE-RSA-AES128-SHA256 DHE-DSS-AES128-SHA256 DHE-RSA-AES128-SHA DHE-DSS-AES128-SHA AECDH-AES128-SHA ADH-AES128-GCM-SHA256 ADH-AES128-SHA256 ADH-AES128-SHA ECDH-RSA-AES128-GCM-SHA256 ECDH-ECDSA-AES128-GCM-SHA256 ECDH-RSA-AES128-SHA256 ECDH-ECDSA-AES128-SHA256 ECDH-RSA-AES128-SHA ECDH-ECDSA-AES128-SHA AES128-GCM-SHA256 AES128-SHA256 AES128-SHA PSK-AES128-CBC-SHA ECDHE-RSA-DES-CBC3-SHA ECDHE-ECDSA-DES-CBC3-SHA SRP-DSS-3DES-EDE-CBC-SHA SRP-RSA-3DES-EDE-CBC-SHA SRP-3DES-EDE-CBC-SHA EDH-RSA-DES-CBC3-SHA EDH-DSS-DES-CBC3-SHA AECDH-DES-CBC3-SHA ADH-DES-CBC3-SHA ECDH-RSA-DES-CBC3-SHA ECDH-ECDSA-DES-CBC3-SHA DES-CBC3-SHA DES-CBC3-MD5 PSK-3DES-EDE-CBC-SHA EDH-RSA-DES-CBC-SHA EDH-DSS-DES-CBC-SHA ADH-DES-CBC-SHA DES-CBC-SHA DES-CBC-MD5 EXP-EDH-RSA-DES-CBC-SHA EXP-EDH-DSS-DES-CBC-SHA EXP-ADH-DES-CBC-SHA EXP-DES-CBC-SHA EXP-RC2-CBC-MD5 EXP-RC2-CBC-MD5 EXP-ADH-RC4-MD5 EXP-RC4-MD5 EXP-RC4-MD5 DHE-RSA-CAMELLIA256-SHA DHE-DSS-CAMELLIA256-SHA DHE-RSA-SEED-SHA DHE-DSS-SEED-SHA DHE-RSA-CAMELLIA128-SHA DHE-DSS-CAMELLIA128-SHA "
Bruce Willman
ITCAM L2 support
Modified on by harrietj 100000VRFW
|
The WRT TCP Status queries can be used to trigger a situation when there are no packets sent from a server.
Here is an example of a situation which triggers if there are TCP packets sent, but zero packets received in a 15 minute period:

Note that the WRT agent must be configured to report 'zero records' for this situation to work,
and that there are performance penalties for enabling this feature.
Zero Record interval is an HTTP/HTTPS configuration, so it does not apply to TCP data.
What it does is to report even if there are not any transactions for the application, transaction, server, or client.
Zero Record interval can be used to trigger a situation when there are zero responses.
Here is a sample configuration excerpt from 'itmcmd config -A t5':
Once a Record is created continue to report on it for the duration of the Zero Record interval.
Report Zero Records for Applications [ 1=Yes, 2=No ] (default is: 2): 1
Once a Record is created continue to report on it for the duration of the Zero Record interval.
Report Zero Records for Clients [ 1=Yes, 2=No ] (default is: 2): 1
Once a Record is created continue to report on it for the duration of the Zero Record interval.
Report Zero Records for Servers [ 1=Yes, 2=No ] (default is: 2): 1
Once a Record is created continue to report on it for the duration of the Zero Record interval.
Report Zero Records for Transactions [ 1=Yes, 2=No ] (default is: 2): 1
Interval (minutes) to report continue to report on zero records
Zero Record Interval (in minutes) (default is: 480):
Bruce Willman
ITCAM L2 support
Modified on by harrietj 100000VRFW
|
The candle_installation.log shows the following error:
libdir is /IBM/ITM/li6243/gs/lib
gskver is gsk7ver
/IBM/ITM/li6243/gs/bin/gsk7ver: error while loading shared libraries: libstdc++.so.5: cannot open shared object file: No such file or directory
Error: Verify Failed
The problem is that the system is missing the required 32-bit system libraries required by GSKit version 8.
- Install the required Linux system libraries
Here is a list of the 32-bit system libraries that are loaded by the ITCAM agents:
/lib/libpthread.so.0
/lib/librt.so.1
/lib/libdl.so.2
/usr/lib/libstdc++.so.6
/lib/libm.so.6
/lib/libgcc_s.so.1
/lib/libc.so.6
/lib/ld-linux.so.2
/lib/libresolv.so.2
/lib/libfreebl3.so
GSKit 8 requires the following libraries:
libstdc++.so.6
libXtst.so.6
ld-linux.so.2
The libs can be installed with commands similar to these:
'yum install libstdc++.so.6'
'yum install libXtst.x86_64'
'yum install glibc-2.12-1.132.el6_5.3.i686'
- Upgrade the Tivoli Framework
You will need this package from Passport Advantage:
IBM Tivoli Monitoring V6.3.0.7 Agents
NOTE: All agents and ITM components on the system will be restarted by the upgrade!
The solution is to upgrade the Tivoli Framework version that is used by the agent.
Here is a tech note describing the process in general terms:
http://www-01.ibm.com/support/docview.wss?uid=swg21673490
The Tivoli Framework used by the ITCAM agent as delivered is 'ax lx8263 Version: 06.23.03.00':
ax IBM Tivoli Monitoring Shared Libraries
lx8263 Version: 06.23.03.00
lx8266 Version: 06.23.03.00
To enable the T6 agent to use GSKit 8, we need to add 'ax li6263 06.30.07.00'.
The generic command to upgrade the Tivoli Framework is:
<MEDIA>/install.sh -h <CANDLEHOME> -q -p <MEDIA>/unix/tf<PLAT>.txt
Here is an example, where the install media is unpacked in /tmp/6.3.0-TIV-ITM-FP0007:
/tmp/6.3.0-TIV-ITM-FP0007/install.sh -h /opt/IBM/ITM -q -p /tmp/6.3.0-TIV-ITM-FP0007/unix/tfli6263.txt
/tmp/6.3.0-TIV-ITM-FP0007/install.sh -h /opt/IBM/ITM -q -p /tmp/6.3.0-TIV-ITM-FP0007/unix/tflx8263.txt
- Install the ITCAM agent
Check <ITM_HOME>/logs/candle_installation.log for errors, and verify that the
upgrade was successful.
After the framework upgrade, the pertinent info is:
[root@nc9053114194 bin]# cat /opt/IBM/ITM/config/gsKit.config
GskitInstallDir=/opt/IBM/ITM/li6243/gs
GskitInstallDir_64=/opt/IBM/ITM/lx8266/gs
[root@nc9053114194 bin]# ./cinfo -i
ax IBM Tivoli Monitoring Shared Libraries
li6263 Version: 06.30.07.00
lx8263 Version: 06.30.07.00
lx8266 Version: 06.30.07.00
gs IBM GSKit Security Interface
li6243 Version: 08.00.50.05
lx8266 Version: 08.00.50.05
Then start the agent to utilize the new Tivoli Framework.
Check the <hostname>_<pc>_*.log to ensure that the GSKit 8 libraries are loaded:
+53FDDF68.0030 /opt/IBM/ITM/li6243/gs/lib/libgsk8ssl.so
+53FDDF68.0030 /opt/IBM/ITM/tmaitm6/li6263/lib/libgsk8cms.so
+53FDDF68.0030 /usr/lib/libstdc++.so.6
+53FDDF68.0030 /opt/IBM/ITM/tmaitm6/li6263/lib/libgsk8sys.so
+53FDDF68.0030 /opt/IBM/ITM/tmaitm6/li6263/lib/kdebe
+53FDDF68.0030 /opt/IBM/ITM/li6243/gs/lib/libgsk8iccs.so
+53FDDF68.0030 /opt/IBM/ITM/li6243/gs/lib/libgsk8kicc.so
+53FDDF68.0030 /opt/IBM/ITM/li6243/gs/lib/N/icc/icclib/libicclib083.so
+53FDDF68.0030 /opt/IBM/ITM/li6243/gs/lib/C/icc/icclib/libicclib080.so
+53FDDF68.0030 /opt/IBM/ITM/li6243/gs/lib/C/icc/osslib/libcryptoIBM080.so.0.9.8
+53FDDF68.0030 /opt/IBM/ITM/li6243/gs/lib/N/icc/osslib/libcryptoIBM083.so.1.0.1
+53FDDF68.0030 /opt/IBM/ITM/li6243/gs/lib/libgsk8dbfl.so
+53FDDF68.0030 /opt/IBM/ITM/li6243/gs/lib/libgsk8valn.so
Modified on by harrietj 100000VRFW
|
Under some circumstances, the Warehouse Proxy Agent does not warehouse and prune historical data files fast enough. Consequently, the size of the short-term history files becomes uncontrolled.
Note: These steps use the Transaction Reporter as an example. The same analysis techniques apply to the other ITCAM for Transactions agents.
On 32-bit platforms, after the size of a short-term history file reaches about 2GB, the Transaction Reporter stops reporting response-time metrics to the TEP (Tivoli Enterprise Portal). To recover, stop the Transaction Reporter, delete the Transaction Reporter short-term history files, and restart the Transaction Reporter.
The size of short-term history files can be monitored with the following steps.
- Add the following KBB_RAS1 trace parameters to the Transaction Reporter configuration:
Unix/Linux: Edit the (<ITM_HOME>/config/to.ini file.
Windows: Use the MTEMS (Manage Tivoli Enterprise Monitoring Services) GUI. Right-click the agent. Select Advanced > Edit Trace Parms...
Note: We use line breaks in the KBB_RAS1 statement for readability only. Do not insert line breaks or carriage returns in the actual KBB_RAS1 statement.
Note: You only include 'KBB_RAS1=' when you edit the agent .ini file directly (example <ITM_HOME>/config/to.ini). When you use the MTEMS GUI to edit the trace parameters, you do not include 'KBB_RAS1='.
KBB_RAS1=ERROR (unit:khdxcpub,Entry="KHD_ValidateHistoryFile" state er)
(unit:khdxhist,Entry="openMetaFile" state er)
(unit:khdxhist,Entry="open" state er)
(unit:khdxhist,Entry="close" state er)
(unit:khdxhist,Entry="copyHistoryFile" state er)
- Restart the Transaction Reporter agent.
- Find the current Transaction Reporter trace logs. For example, the Transaction Reporter trace log filename is <hostname>_to_<timestamp>.log.
- Filter the Transaction Reporter trace log using the string "KHD_ValidateHistoryFile".
For example, with the Transaction Reporter, records similar to the following are generated:
(50212426.0007-A:khdxcpub.cpp,1383,"KHD_ValidateHistoryFile") History file /opt/Tivoli/ITM61/aix533/to/hist/AGGREGATS row length is 3404, size is 1477336.
- Filter the output from the previous step by using the name of the short-term history file. For example, for the Transaction Reporter, use ‘INTERACTN’ or ‘AGGREGATS’.
See Implementing Historical Data Collection in ITMhttp://www.ibm.com/support/docview.wss?uid=swg21516594
- Filter out the numeric value of the size. For example, where size is 1477336, filter out the value 1477336. Here, the size is the current size of the short-term history file.
- Load these numeric values into your favorite graphing application or spreadsheet.
Example: Initially, the current size of the short-term history file continues to increase and then level off. In this example, the current size of the short-term history file stabilizes between 12MB and 14MB.

Warehousing Optimization Messages
The Warehouse Proxy Agent is optimized to use a file pointer to mark the starting point for the agent to resume processing history records. For example, the message 0 skipped means that the agent started at the beginning of the data.
(502122FC.0002-D:khdxhist.cpp,1724,"close") /opt/Tivoli/ITM61/aix533/to/hist/AGGREGATS - 434 rows fetched, 0 skipped
When the number of skipped records is not zero, it means that the Warehouse Proxy Agent skipped over records that it processed previously. For example, the message 434 skipped does not mean that data is ignored or lost. The message simply says that the Warehouse Proxy Agent skipped over 434 rows that it had processed previously.
(5021267F.0002-D:khdxhist.cpp,1724,"close") /opt/Tivoli/ITM61/aix533/to/hist/AGGREGATS - 1325 rows fetched, 434 skipped
Modified on by harrietj 100000VRFW
|
Sometimes the ITCAM Transaction Reporter Topology view displays only unstitched nodes, similar to this:
![]()
If you look carefully at the bottom of the veiw, there is a warning message - 'Topology incomplete: check topology properties'.
The remedy for this problem is to edit the properties for the view.
Right click in the view, and select Properties:
![]()
In the 'Number of Interactions to display', select 'Display all':
![]()
Save the changes, and the topology should now be stitched:
![]()
Bruce Willman

Subscribe and follow us for all the latest information directly on your social feeds.
Modified on by harrietj 100000VRFW
|
You can enable or disable content capture in the T6 agent configuration. However, the AMC Editor profile also has options that allow you to enable or disable content capture. Why is it to your advantage to enable or disable this feature in the agent configuration?
The AMC Editor profile overrides the content capture option in the T6 configuration.
The AMC Editor profile allows you to enable or disable content capture for the scripts associated with the profile, or use the value set in the T6 configuration (Agent Default).
So if you set the profile to Agent Default and you enable this feature in the T6 configuration, you would capture content images on VP failures.
If the content capture option is disabled in the agent configuration, then the profile setting (true or false) determines whether you capture images on VP failures.
Please note that captured images have a default lifetime (default is 8 hours). The maximum number of captures is 1000. So if you are trying to display an image that is over 8 hours old or is not included in the 1000 most recent images, then you would not be able to display the image.
According to the V7.4.0.1 Administrator's Guide, the content capture feature is only available for RFT scripts and for RPT scripts that use HTTP protocols.
Also see:
Robotic Screen Capture
https://www.ibm.com/support/knowledgecenter/SS5MD2_7.4.0.1/com.ibm.itcamt.doc/rt/Workspaces/T6_RRT/wksp_t6_screencapturerobotic.html
Configuring Content Capture data retention
http://www.ibm.com/support/docview.wss?uid=swg21593345
ITCAM4Tx : RRT Robotic Screen Capture is a Content Capture utility
http://www.ibm.com/support/docview.wss?uid=swg21438339
RRT Robotic Content Capture environment checks
http://www.ibm.com/support/docview.wss?uid=swg21426717
Modified on by harrietj 100000VRFW
|
Where can I see it
If you want to see topologies there are two choices:
1) Use IBM Performance Management (v8.1.3) - either on-Prem or on-Cloud. Use IPM's web based interface.
2) Use ITCAM for Transactions v7.4.0.1 and view it via the TEPS
The export functionality is for SCAPM to provide additional links between its workspaces. SCAPM itself does not have ability to display topologies.
Where can I not
Transaction Tracking displays its own data as a topology (ie via TEPS/ITM). The knowledge center document below talks about importing Transaction Tracking data into SCAPM to allow SCAPM to understand server/application relationships, so it's UI can be enriched with additional links (eg from SCAPM's view of Websphere App server, jump to its view of MQ). SCAPM does NOT use this data to display a topology, as SCAPM (as a product) has no topology view.
Topologies can be displayed in Application Performance Management on Cloud.
simply indicates that APM v8.1.3 can display transactions topologies. It does not imply it takes this information from ITCAM for Transactions v7.4.0.1.
Products in discussion in the document below:
* SmartCloud Application Performance Monitoring (aka SCAPM v7.7)
* Transactions Tracking (TT) as a component of ITCAM for Transactions 7.4.0.1
* Application Performance Management (on Cloud), aka IBM Performance Management (IPM) aka IBM Application Performance Manage (IAPM). There are actually two "versions", the on-Premise and the on-cloud version. Functionally they are very similar, it is a delivery model difference.
* Performance Management in the cloud -> https://www.ibmserviceengage.com/application-monitoring/learn
* Performance Management on premise is available through Passport Advantage
* IPM has its own tracking functionality, and can display topology. This is Performance Management V8. The current version is APM v8.1.4.

Subscribe and follow us for all the latest information directly on your social feeds:

Modified on by harrietj 100000VRFW
|
ITCAM for Transactions 7.4.0.2 supports RPT 9.1
Check this technote for details on this package http://www.ibm.com/support/docview.wss?uid=isg400003574
There are special considerations based on the version of ITM installed.
ITCAM for Transactions 7.4.0.1 supports RPT 9.0 interim fix: 7.4.0.1-TIV-CAMRT-IF0008 is required. We also recommend the security updates provided with
7.4.0.1-TIV-CAMRT-IF0009. There are later fixpacks as well.
Here are the part numbers for RPT 9. Use IBM Installation Manager to upgrade RPT after installation.
![]()

Subscribe and follow us for all the latest information directly on your social feeds:

Modified on by harrietj 100000VRFW
|
You must not have more than one AMC in a given ITM environment. If you have more than one online AMC in your environment, AMC profiles and profile changes are not distributed to agents, and robotic scripts do not run.
Use the tacmd listSystems command to list all instances of the AMC agent in your ITM environment. The AMC agent is identified by the product code T3 and is often referred to as the T3 agent.
The listSystems command searches all of the managed systems connected directly to the hub TEMS and indirectly through remote TEMS servers.
To determine if you have more than one T3 agent in your ITM environment:
- Logon to the hub TEMS.
- Use the tacmd login command to log onto the hub TEMS service.
ITM_HOME\BIN\tacmd login -u sysadmin -p <sysadmin-password> -s <full-hub-tems-server-name>
Validating user...
KUIC00007I: User sysadmin logged into server on https://<hub-tems-server-name>:<port>. - After you have established a connection to the hub TEMS service, enter the following tacmd listSystems command.
Windows:
ITM_HOME\BIN\tacmd listSystems | find ":T3"
Unix/Linux:
ITM_HOME/bin/tacmd listSystems | grep \:T3
TIP: Filter the command output to list the occurrence of T3 agents. Use the Find filter under Windows, and use the grep filter under Unix/Linux.
Note: Be sure to use the filter :T3 (with the colon). The filter T3 (without the colon) lists all of the applications known to the AMC.
Sample Output
C:\IBM\ITM\BIN\tacmd listSystems | find ":T3"
NC9053120203:T3 T3 07.40.01.00 Y
This example proves that there is only one AMC in this ITM environment.
- The agent is online (indicated by the "Y" in last column).
- The agent maintenance level is ITCAM for Transactions 7.4 FP01 without interim fixes.
- The agent resides on server NC9053120203. Note that this is the short hostname.
Hub TEMS on z/OS
If your hub TEMS resides on z/OS, launch the tacmd login command from a supported distributed platform. Connect to the hub TEMS on z/OS. Then, launch the tacmd listSystems command from the distributed platform to return agent information.
Modified on by harrietj 100000VRFW
|
When a verification point failure occurs in RPT (Rational Performance Tester) scripts, the default behavior is to report the error and then continue to the next page element (transaction). Starting in RPT 8.2, you can select the way you want your RPT scripts to handle verification point errors. Normally, you want the script to exit, but other options are available that may be more appropriate in some scenarios.
This blog explains how to set these page element exit policies in your RPT script.
Users who are playing back RPT scripts under the Robotic Response Time agent in ITCAM for Transactions V7.4 will find this presentation especially useful.
1) Display your script in RPT test mode. Right-click the parent node and select Verification Points > Enable Page Title VPs from the pop-up menus.
2) A pop-up message asks if you want to globally apply page title verification points to your script. Click OK.
3) RPT acknowledges that it has added the verification points. Click OK.
4) One at a time, select the top-level page elements. Then, click Edit Properties.
5) In this example, the default action is to continue if the verification point is detected. When you use this default for each page element, each page element generates a verification point failure if the web page is not accessible – for example, if the application or the application server is offline. To change the default behavior, click Change.
6) Note the different options available for continuing or ending script execution. If you select Exit Test, you see one verification point failure in the scenario where the web page or the application server is down. In most cases, this is the best selection, because you don’t want to flood Omnibus or the Robotic Script Verification Point Failures workspace with failures.
7) Notice that the default action is now to exit the test. Click OK.
8) Repeat for each of the high-level page elements in your script. In this example, there are 5 top-level page elements.
9) Run a test in RPT to compile the script. If no errors are reported in the RPT test log, export the script to the AMC (Application Management Console).

Modified on by harrietj 100000VRFW
|