IV27944: LARGE AMOUNT OF MEMORY CONSUMED BY ITM WINDOWS OS AGENT KNTCMA.EXE PROCESS

IBM Tivoli Monitoring: Windows (R) OS Agent 6.2.3.2-TIV-ITM_WIN-IF0001
IBM Tivoli Monitoring: Windows (R) OS Agent 6.2.2.9-TIV-ITM_WIN-IF0001
IBM Tivoli Monitoring: Windows (R) OS Agent 6.2.2.4-TIV-ITM_WIN-IF0004

APAR status

• Closed as program error.

Error description

• ITM Windows OS agent uses excessive memory when running
  situations that evaluate against the "Event Log" attribute
   group (KNT.NTEVTLOG).
  
   Large amounts of memory can be consumed quickly, or memory
   use may grow over time indicating a memory leak against the
   kntcma.exe process.  This is reflected in the "virtual bytes"
   performance counter (perfmon) value increasing for kntcma.
  
   Environment where Virtual Memory use increases even with
   NT_LOG_THROTTLE=1 set:
  
   Microsoft Windows Server 2008 R2 Enterprise 64-bit
   ITM OS agent for Windows 6.22 FP9 - WIX64
   Large event log size on the OS - example is Security event
    log for Domain controller with over 250 million records
   ITM situation formula using "Event Log" attribute group
    that does not specify the Log Name as part of the formula,
    which results in the situation processing against all event
    logs on the system.
  
  Ex:
  tacmd viewSit -s Event_Log_Example
  Name                   : Event_Log_Example
  Full Name              :
  Description            :
  Type                   : Windows OS
  Formula                :
   *IF ( ( *VALUE NT_Event_Log.Event_ID *EQ 2011 *AND *VALUE
  NT_Event_Log.Source *EQ Srv )
     *OR ( *VALUE NT_Event_Log.Event_ID *EQ 3013 *AND *VALUE
  NT_Event_Log.Source *EQ Rdr ) )
  Sampling Interval      : 0/0:0:0
  
  Debugging:
   Gather concurrent KBB_RAS1 logging for Kntcma.exe process along
   with OS perfmon outputs showing memory usage.
  
  
  
  In NT agent KNTENV file set:
  KBB_RAS1=ERROR (UNIT:KNT ALL) (UNIT:KRA ALL) (UNIT:KNZ ALL)
   (UNIT:KNL ALL)
  
  To enable perfmon logging:
    start the perfmon interface
    in the navigator on the left open "performance logs and
  alerts"
    right-click on "Counter Logs" and select "New Log Settings
  ..."
    enter a name (e.g. ITMagentmemory_data)
    In the "General" tab, select "Add counters"
    Select "Process" as performance object
    Select "Virtual Memory" attribute as "select counters from
  list"
    Select the kntcma process from the list of instances.
    click on the "add" button and then on the "close" button
    still in the "General" tab, enter 1 sec as the interval for
  samples
    Take note of the directory and log name in the
     "Current Log file name" field.
    In the "Log Files" tab, in the "Log file type" pull down menu
     specify "Text File (Tab delimited)" as type
    In the "Schedule" tab, under "Start Log", select the Manually
     radio button.
    Click the "OK" button and answer "yes" to any question.
    The agentmemory_data log is listed on the right side of the
     window.
    Right click on the log and select "Start"
  
  Monitor the memory usage until it increases an additional 50MB
  more for the kntcma.exe process, then return to the perfmon
  interface and right-click on the log file and select "Stop".
  
  
  Gather "pdcollect" output for ITM OS agent RAS1 logs along with
  the perfmon log file for review.
  
  
  Additional Keywords:
   DCF 1586284
  

Local fix

• Stop situations evaluating against Event Log attributes
   and confirm if memory consumption is resolved.
  
   Modify any Event Log situation to add the Log Name attribute
   to the formula so the situation is not evaluated against all
   event logs on the system.  In the below example, the
   formula was modified to use "System" from the pulldown menu
   in the Situation Editor.
  Ex:
  tacmd viewSit -s Event_Log_Example
  Name                   : Event_Log_Example
  Full Name              :
  Description            :
  Type                   : Windows OS
  Formula                :
   *IF ( ( *VALUE NT_Event_Log.Event_ID *EQ 2011 *AND
           *VALUE NT_Event_Log.Source *EQ Srv *AND
           *VALUE NT_Event_Log.Log_Name *EQ System )
     *OR ( *VALUE NT_Event_Log.Event_ID *EQ 3013 *AND
           *VALUE NT_Event_Log.Source *EQ Rdr *AND
           *VALUE NT_Event_Log.Log_Name *EQ System ) )
  Sampling Interval      : 0/0:0:0
  

Problem summary

• Problem Description: LARGE AMOUNT OF MEMORY CONSUMED BY ITM
  WINDOWS OS AGENT KNTCMA.EXE PROCESS
  
  Problem/Problem Summary: In Windoms 2008, ITM Windows OS agent
  uses excessive memory when running situations which evaluate
  against the "Event Log" attribute group (KNT.NTEVTLOG).
  
  Large amount of memory can be consumed quickly, or memory
  usage may grow over time, indicating a memory leak against the
  kntcma.exe process. The memory leak becomes evident when
  many events are being received by the system.
  
  The problem is associated to the event XML information appended
  to the "description" attribute of Event Log attribute group.
  By displaying the list of events in the TEP user iterface,
  extra data is appended with an XML tag such as <EventData>
  <Data> or <UserData> or similar ones.
  

Problem conclusion

• The instructions causing the memory leak have been amended.
  
  The fix for this APAR is contained in the following maintenance
  package:
  | interim fix | 6.2.3.1-TIV-ITM_WIN-IF0002
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  ITM AGENT WINDO

• Fixed component ID

  5724C040W

Applicable component levels

• R622 PSY

     UP

• R623 PSY

     UP

• R610 PSN

     UP

• R620 PSN

     UP

• R621 PSN

     UP