Technical Blog Post
Performance issues Part 5 TEMS
Performance issues: TEMS
This can be a HUB or an RTEMS. The issues can be reported as disconnects, hangs, crashes, even the TEPS not responding; so this is a large topic.
There are some configuration steps to be aware of that can help avoid some performance issues:
1) On Unix and Linx the ulimit value of nofiles (file descriptors) needs to be at least 8192 , this is sometimes missed but can make a big difference to performance.
Like any Operating system value, it also depends on what , if anything, else is running on the machine. Therefore any values other software needs should also be taken into account.
As stated in the title blog for this series there is a section in the Installation guide on performance tuning:
There is also a blog on tuning for high workloads, make sure you check the update at the end! :
then there is this blog:
which does discuss the KDE1_STC_RXLIMITEXCEEDED value that reduces the amount of data that is requested from an agent.
2) The number of agents per TEMS, this should be limited to around 1500 after that there are often issues with the RTEMS performance, of course it also depends on the type and number of situations used.
If a RTEMS is under a heavy load it often shows up with disconnects between it and the HUB, as it is too busy to respond with heartbeats in the correct time span.
3) As already stated in the initial blog in this series here is tuning information in the Install and Administrators guide, make sure that this is read and decisions are made correctly for the environment that is being run.
4) Avoid creating duplicate agents; all agents should have a unique name for its ip address. Make sure that if servers are created via a clone, or agent configuration files are copied the hostnames are unique.
However even once a TEMS is configured correctly there can still be issues and there are tools out there to help you with checking that the environment is healthy and help pick up any problems.
The first of these is the datahealth:
then there is the TEMSAudit
then if you get reports of duplicate agents in your datahealth report you should run this:
Once you have the names and ip addresses of duplicate agents, then it is manual process to find out the issue.
Although often it is is due to CTIRA_HOSTNAME being set the same over a number of agents as the configuration files have been copied, or the servers have been set up as clones.
5) There is a separate blog in this series (part 2) that talks about situations, as these can cause load at the TEMSes due to how the filters in the situation are written, as well as the number that are sent.
The datahealth report will also check for some issues with situations.
Subscribe and follow us for all the latest information directly on your social feeds:
|Academy Twitter :||https://goo.gl/HnTs0w|