Performance Configurations

2 likes Updated 12/15/14 3:29 PM by TechSalesDudeTags: None

To take advantage of the speed and scalability offered by Tivoli Endpoint Manager (TEM), it is often necessary to tune the settings of your TEM deployment. Performance tuning is especially important when your TEM Server does not meet the recommended TEM Server requirements or if you have a medium to large TEM deployment.

To increase performance...

 

Use SQL Server instead of SQL Server Desktop Edition (MSDE) or SQLExpress

           
  Component:       TEM Server Database
           
  Description:      

The TEM Server installer includes an installer for SQL Server Express. Resource usage and connections are restricted by design in SQL 2005 Express, and therefore is not recommended for production use. The TEM Server components use several database connections and performs best when SQL Server has access to ample resources. This can corrected by using SQL Server Standard Edition (or Enterprise Edition) instead of SQL Server Express.

             
  Likely Impact:       High for deployments of all sizes with multiple TEM Console operators.  
             

 

Use TEM Relays to reduce total network load

             
  Component:         Network bandwidth
  Description:        

By default, the TEM Server downloads patches, upgrades, etc. from the Internet and then the TEM Clients download the files from the TEM Server. If the TEM Server is in a separate geographic location from a group of TEM Clients with a slower network connection, each TEM Client must download the file over the slow link from the TEM Server.

For example, imagine that there is a company with a main office in LA with the TEM Server and a remote office in NY with 500 TEM Clients connected using a T1. If Windows 2000 SP4 (120 MB download) is deployed to each of the TEM Clients in NY, there will be a total of 60 GB of network traffic, which can easily overwhelm the T1 connection (this situation would be much worse if there is a slower connection of 256 kbps or even 56 kpbs), which is used for many other activities.

To avoid this unattractive use of bandwidth, a TEM Relay should be installed on a computer in NY and the TEM Clients in NY should be configured to download from the TEM Relay. Using this configuration, the TEM Relay will download and cache each file from the TEM Server and the TEM Clients will download directly from the TEM Relay. In the previous example, the total network traffic across the link from LA to NY would be reduced from 6 GB to 120 MB. Note that the TEM Relays will also group and compress communications other than downloads from the TEM Clients to the TEM Server providing even more bandwidth savings.

For more information about TEM Relays, see TEM Relays.

  Likely Impact:         Very high for any size deployment with remote locations with slow bandwidth.

 

Use TEM Relays to reduce TEM Server load

             
  Component:         TEM Architecture
             
  Description:        

For all but the smallest TEM deployments (< 500 TEM Clients), a primary TEM Relay should be set for each TEM Client even if they are not in a remote location. The reason for this is that the TEM Server performs a lot of tasks including gathering new Fixlet content from the TEM Servers, distributing new Fixlet content to the TEM Clients, accepting and processing reports from the TEM Clients, providing data for the TEM Consoles, sending downloaded files (which can be quite large) to the TEM Client, and much more. By using TEM Relays, the burden of communicating directly with every TEM Client is effectively moved to a different computer (the TEM Relay computer), which frees up the TEM Server to do other tasks. If TEM Relays are not used, you may see that performance degrades significantly when an action with a download is sent to the TEM Server and you may even see errors.

Setting up TEM Relays in the appropriate places and correctly configuring the TEM Clients to use them (either by letting the TEM Client auto-select their closest TEM Relay or by manually configuring the TEM Clients to use a specific TEM Relay) is the most important and highest impact performance change.

For more information about TEM Relays, see TEM Relays.

             
  Likely Impact:        
  Small Deployments (<3000 Clients) High  
  Medium Deployments (<15,000 Clients) Very high  
  Large Deployments (>15,000 Clients) Very high  

 

Lower heartbeat frequency 

             
  Component:         TEM Server
             
  Description:        

By default the TEM Clients send to the TEM Server on a regular interval a message known as a "heartbeat". When the TEM Clients send in a heartbeat to the TEM Server, they will update their "Last Report Time" property along with any other properties that have changed since the last heartbeat. In medium to large TEM deployments, processing the heartbeats can consume significant TEM Server resources. To ensure optimal performance, the interval between two consecutive heartbeats should be raised from the default 15 minutes to 1 hour or even 2-6 hours for larger TEM deployments. The heartbeat interval can be changed under the File > Preferences menu in the TEM Console.

             
  Likely Impact:        
  Small Deployments (<3000 Clients) Low  
  Medium Deployments (<15,000 Clients) Low  
  Large Deployments (>15,000 Clients) Medium  

 

Lower TEM Console refresh frequency

             
  Component:         TEM Console
             
  Description:        

When the TEM Console is being used, the TEM Console will query the TEM Server database and cache the results locally. The cache is updated according to the TEM Console refresh period. The more TEM Clients, the more data is transferred to the TEM Console using database resources and network bandwidth. The TEM Console refresh period should be raised to from its default of 15 seconds to 30 seconds, 60 seconds, or even 180 seconds for large deployments with lots of simultaneous TEM Console users. The refresh period can be changed under the File > Preferences menu in the TEM Console (note that this setting is per TEM Console).

Note: You can use the TEM Administration tool to set a global minimum refresh rate for all your TEM Console operators. You should set this to a higher number (30-60 seconds might be a good starting point) if you are experiencing slowness in the TEM Console.

             
  Likely Impact:        
  Small Deployments (<3000 Clients) Low  
  Medium Deployments (<15,000 Clients) Medium  
  Large Deployments (>15,000 Clients) Very high  

 

Configure virus scanners not to scan TEM Server / TEM Relay folder

             
  Component:         TEM Server / TEM Relay
             
  Description:        

The TEM Server and TEM Relay's normal operations involve creating and processing a lot of temporary files. This activity is essential for good performance of TEM, but can be slowed down dramatically or cause other anomalies if a virus scanner is scanning each file. To address this issue, configure your virus scanner on the TEM Server and TEM Relay computers to exclude the TEM Server folder and all subfolders (default is "C:\program files\bigfix enterprise\bes server") or the TEM Relay and its subfolders (default is "C:\program files\bigfix enterprise\bes relay"). Refer to instructions from your virus scanner for more information on how to set this exclusion rule.

             
  Likely Impact:        
  Small Deployments (<3000 Clients) Low  
  Medium Deployments (<15,000 Clients) Medium  
  Large Deployments (>15,000 Clients) Medium  

 

Do not use file indexing or file compression on TEM Server / TEM Relay computers

             
  Component:         TEM Server / TEM Relay
             
  Description:        

The TEM Server and TEM Relay's normal operations involve creating and processing a lot of temporary files. This activity is essential for good performance of TEM, but can be slowed down dramatically if Windows file indexing is turned on or if the drive is set to use file compression. If these computers have indexing or compression enabled, you should disable them.

             
  Likely Impact:        
  Small Deployments (<3000 Clients) Low  
  Medium Deployments (<15,000 Clients) Medium  
  Large Deployments (>15,000 Clients) Medium  

 

Configure RAID cache performance

             
  Component:         TEM Server
             
  Description:        

The hard drive performance of the TEM Server is very important to the overall system performance. If improperly configured, RAID controllers can cause severe performance problems. Specifically, some on-board RAID controllers for hard drives come configured with the hard drive cache set to 100% Read/0% Write performance. This configuration will cause very poor performance because all hard drive writes will be very slow. The optimal configuration is 50% Read/50% Write performance. Refer to your RAID instructions for how to set this. If the on-board controller cannot be configured for 50% Read/50% Write performance, then you should purchase an off-board controller. Note: May not be applicable to most Solid State Drive based storage. Warning: Using RAID hard drive cache for writes increases the likely hood of data loss during power outages.

             
  Likely Impact:        
  Small Deployments (<3000 Clients) Medium  
  Medium Deployments (<15,000 Clients) Very high  
  Large Deployments (>15,000 Clients) Very high  

 

Configure RAID Array for Read/Write Speed

             
  Component:         TEM Server
             
  Description:        

The hard drive performance of the TEM Server is very important to the overall system performance. If improperly configured, RAID controllers can cause severe performance problems. The TEM Server will both read and write lots of data to disk and requires very low I/O latency for best performance. For this reason, RAID 10 is the best choice for RAID configuration, and locally-attached SSD or other low latency disk should be considered.

             
  Likely Impact:        
  Small Deployments (<3000 Clients) Medium  
  Medium Deployments (<15,000 Clients) High  
  Large Deployments (>15,000 Clients) Very high  

 

Move TEM Server program files to different drive than SQL Server database

             
  Component:         TEM Server
             
  Description:        

The TEM Server performance is affected heavily by the write performance of the computer. The TEM Server writes many files to disk and also writes data to the database at the same time. If the TEM Server's program folder is on a separate physical drive than the SQL Server database, the hard drive writes can be done in parallel, which can lead to speed increases.

             
  Likely Impact:        
  Small Deployments (<3000 Clients) Medium  
  Medium Deployments (<15,000 Clients) Medium  
  Large Deployments (>15,000 Clients) High  

 

Use high speed network connections for TEM Server/TEM Relays

             
  Component:         TEM Server/TEM Relays
             
  Description:        

The TEM Server and TEM Relay's performance are partially bound by bandwidth -- especially for large file downloads. By using gigabit network connections or dual network cards on the TEM Server and TEM Relays, you can effectively increase the bandwidth and increase the speed of the TEM Clients downloading and reporting.

             
  Likely Impact:        
  Small Deployments (<3000 Clients) Low  
  Medium Deployments (<15,000 Clients) Low  
  Large Deployments (>15,000 Clients) Medium  

 

Use TEM Management Rights

             
  Component:         TEM Console
             
  Description:        

When the TEM Console connects to the TEM Server computer, all of the data about the TEM Clients, the Fixlet messages, actions, etc. are sent from the TEM Server to the TEM Console. The amount of information transferred grows as the number of TEM Clients grows and can become quite large for medium to large deployments. By using the management rights feature to restrict the number of TEM Clients managed by each TEM Console operator, the bandwidth between the TEM Server and TEM Console can be reduced significantly because the TEM Console will receive information about only a subset of the TEM Clients.

             
  Likely Impact:        
  Small Deployments (<3000 Clients) Low  
  Medium Deployments (<15,000 Clients) Medium  
  Large Deployments (>15,000 Clients) High  

 

Always use the "Stagger action start times..." feature

             
  Component:         Network
             
  Description:        

When the TEM Console operator takes an action, there is an option under the "Execution" tab to "Stagger action start times over X minutes to reduce network load". If this setting is not used, the targeted TEM Clients will all simultaneously attempt to execute the action, which usually involves downloading a file from the TEM Server (or TEM Relay). Not only will this put a large load on the TEM Server, but you may also place a significant load on total network bandwidth (this is especially a problem if TEM Relays are not used). To avoid this problem, always set the "Stagger action start times over..." setting. Each client will then select a random time within that time interval to run the action. You should set the number higher (120 minutes) if many TEM Clients are going to download a large file and set it lower (30 minutes) when targeting only a few TEM Client computers or when deploying smaller patches.

             
  Likely Impact:        
  Small Deployments (<3000 Clients) Low  
  Medium Deployments (<15,000 Clients) Medium  
  Large Deployments (>15,000 Clients) High  

 

Use more powerful TEM Console computers

             
  Component:         TEM Console
             
  Description:        

The system requirements for the TEM Console computer increase substantially for medium to large deployments. If you notice that the TEM Console is running very slowly, using a lot of memory, or constantly causing HD access, you should use the TEM Console on a more powerful computer. In particular, adding more memory can significantly improve the performance of the TEM Console. The TEM Console requirements are available here.

             
  Likely Impact:        
  Small Deployments (<3000 Clients) Medium  
  Medium Deployments (<15,000 Clients) High  
  Large Deployments (>15,000 Clients) Very high  

 

Close unnecessarily open actions

             
  Component:         TEM Client
             
  Description:        

Actions are either in the Open, Closed, or Stopped state. Open actions are sent to TEM Clients so that the TEM Client can determine if it is supposed to run the actions. Each open action puts a small additional load on the TEM Client. If there are many open actions, the TEM Client will be a little slower to respond to newly relevant Fixlets/actions. It is good practice to close open actions if there is no need to have them open. As a reference point, over 10,000 open actions is a very large number of open actions; 5,000 is pretty large; 1,000 is a little high; 200 should be no problem.

             
  Likely Impact:        
  Small Deployments (<3000 Clients) Low  
  Medium Deployments (<15,000 Clients) Low  
  Large Deployments (>15,000 Clients) Medium  

 

Delete closed/expired actions

             
  Component:         TEM Console
             
  Description:        

The TEM Console will list every action that has been taken including every result from every TEM Client. As the number of actions grows large, the TEM Console will use more memory to load the actions and the cache load/write times increase. You can right click and delete old actions, which will mark them as deleted in the database. Once they are marked as deleted, the TEM Console will no longer load them saving memory and load time. The actions will continue to be in the database after they are deleted, but they will not be accessible to the TEM Console or to web reports.

             
  Likely Impact:        
  Small Deployments (<3000 Clients) Low  
  Medium Deployments (<15,000 Clients) Low  
  Large Deployments (>15,000 Clients) Medium  

 

 

Database Cleanup

Component:

Database

 

Description:

After enough time passes, obsolete data in the database may eventually accumulate to the point that the performance of query and TEM operations degrades dramatically. To clean out the database: First, back up the BFEnterprise database. Then, run the computer remover tool and the audit trail cleaner tool on a recurring basis to cleanout old computer and audit trail data within the database for old deleted computers and content.

Computer Remover Tool

Audit Trail Cleaner Tool

Note: Before doing this, determine if your organization requires you to preserve old data in the database for audit trail purposes. If so, back up the database and save the backup in in the event an audit needs to be performed later on.

After the tools have been run to delete out old data perform a database shrink operation on the BFEnteprise database.

 

Likely Impact:

Small Deployments (<3000 Clients) Low

Medium Deployments (<15,000 Clients) Medium

Large Deployments (>15,000 Clients) High