Sitworld: AOA Critical Issue - Duplicate Agent Names
Version 1.00000 - 8 October 2018
John Alvord, IBM Corporation
In August 2014, the Database Health Checker began running at IBM ECUREP as an Analysis On Arrival task on each incoming hub and remote TEMS pdcollect. Since then TEMS Audit and Event History Audit reports have been added. The reports are very useful for by identifying known error condition and thus speeding ITM diagnosis of issues. Each of the tools can be run by any customer, but the AOA reports are not immediately visible. Any customer could ask for them but not being visible no one ever asks. At the same time the reports have become more complex and challenging to digest.
With a recent change, the process has been extended to create a short list of critical issues which will automatically be added to the S/F Case or PMR as a short email text. That creates visibility for critical issues. This document presents one specific critical issue - Duplicate Agent Name cases
Please note that the conditions identified may not be the issue the problem case was opened for. For example one recent case was a FTO hub TEMS switch to backup that was unexpected. After close study, the major issues was mal-configured agents including duplicate name cases, Virtual Hub Table Update floods and several other items. There are also rare cases where a report will be produced concerning an obsolete TEMS that is definitely installed but not in action use. In that case the report could be ignored - although uninstalling the TEMS would be a good idea.
Getting more information
If you are viewing this document as an customer working with IBM Support, you are welcome to request copies of the Analysis On Arrival reports if they are available. Be sure to mention the unpack directory from the AOA Critical Issue report.
TEMS Audit - temsaud.csv [any hub or remote TEMS]
Database Health Checker - datahealth.csv [any hub TEMS]
Event History Audit - eventaud.csv [any hub or remote TEMS]
There are cases when no report is generated. Sometimes that means there were no advisories. TEMS Audit is not produced when the relevant log files cannot be identified. Database Health checker is run but skipped if it appears to be a remote TEMS. Event History Audit and Database Health Checker are not run if there are errors detected in the table extract process.
Visit the links above to access the AOA programs if you want to run the AOA programs at your own schedule.
Duplicate Agent Name cases
temsaud.crit: $idupcnt duplicate agent name cases - see TEMSREPORT069 Report
ITM depends on each agent having a unique name. When that is violated the TEMS can become unstable and the TEPS experiences severe performance problems. Even worse in some ways, the agents affected are not being properly monitored since only one at a time can report conditions. On a more prosaic note, people using the event data often waste considerable time because an event on one agent, when investigated, shows a false positive - not a problem. However there was a real problem but it was for another agent with the same name on another system.
You can make progress by asking for the full TEMS Audit report. You can also use Sitworld: TEPS Audit which report on duplicate agents the TEPS is aware of.
You also need to discover how the duplicate agents are being created and change the process so that duplication is avoided.
You can also involve IBM Support because there are extended TEMS tracing which will show other cases - hidden within the remote TEMS.
This documents explains how to handle duplicate agent name cases.
Note: 2018 - Home Grown Meyer Lemons