Technical Blog Post
Windows OS Agent High Memory usage
Oh no, is this really another blog article talking about high CPU or high memory usage by Windows OS Agent ?
There is indeed a lot of literature about this matter, I can mention the ones I think cover 99% of the
So if you are experiencing similar issues, you can have a look at the above material as it explains why and when the kntcma process can consume high amount of cpu and virtual memory.
To make story short, we know from experience it can happen for:
- Processing of Windows Event log entries
- Wrong/Inefficient formula for windows Event Log situations
- High workload caused by a huge number of situations/historical collections
- Corrupted PerfMon counters
- Problems with watchdog internal modules.
The above articles cover all these scenarios where KNTCMA process shows high CPU or high memory, but it seems so far we missed another one that can be quite common:
the size of the historical binary files.
Let's suppose you enabled historical collection for attribute groups that collects a meaningful amount of data on each interval, for example NTPROCESS (Process attribute group).
If, for any reason, the agent is not able to export the binary data to WPA for long time (weeks or months), the historical data is kept into the local binary file,
and so it may become really huge in size.
In this case, depending on the size of the bigger historical binary file, the agent may refuse to start or, if it starts, you can notice it quickly allocates an abnormal amount of
virtual memory (even some GB).
Also, let's suppose all is fine with WPA but the agent is not able to export historical data anyway.
The RAS1 log of the agent will show error messages like:
khdxdacl.cpp,685,"routeExportRequest") sourceObject open() failed, status = 66, for object xxxxx
where xxxxx varies through many different NT performance objects (NT_System, NT_Memory , NT_Logical_Disks etc).
This failure is caused by a corruption of the file khdexp.cfg, as explained in this technote:
Technotes was created for TEMS, but it is also applicable to agents.
The problem prevents the export of historical files and as a consequence also in this case the size of the files can growth indefinitely.
From user perspective, both the above scenarios result in an unexpected high allocation of virtual memory from KNTCMA process and also a meaningful
consumption (12-20%) of CPU.
Unless you are interested in the content of the historical binary files, the quickest solution is to:
1) Stop the agent
2) Make a backup copy of folder <ITMHOME>\TMAITM6_x64\logs\History\KNT into another location
3) Delete ALL the files from folder <ITMHOME>\TMAITM6_x64\logs\History\KNT
4) Delete psit_Primary_<hostname>_NT.str file from <ITMHOME>\TMAITM6_x64 directory.
5) Restart the agent
Once the agent restarts, it will generate a new khdexp.cfg file and new historical binary files.
If also the issue that prevent communication with WPA has been fixed, the export of the historical data will work fine and the size of the binary file
will be under control.
And as result, also memory allocation for KNTCMA process will be back to normal.
If you are interested in the content of old binary files, you can process the backup copies manually, exporting the data using tools like
krarloff, and then possibly inserting the rows into the related TDW table using manual SQL queries.
These two articles can help with this task:
Hope it helps
Subscribe and follow us for all the latest information directly on your social feeds:
|Academy Twitter :||https://goo.gl/AhR8CL|