This blog promotes knowledge sharing through experience and collaboration. For more product information, visit our WebSphere Commerce CSE page. For easier navigation, utilize the Categories to find posts that match your interest.
Throttle for Damage Control
In this post I discuss the concept of throttling, and how it can be used to bring stability to a site that is being overloaded with traffic.
Throttling puts a cap on the number of shoppers that can reach the servers. The idea is simple: If the site can handle only 50% of the load it's receiving, or otherwise it completely fails/hangs, it is better to handle that 50% than to fail for everybody. For a site experiencing performance problems, such as 100% CPU, or when a 3rd party service is overloaded, throttling can help restore stability.
Throttling is used to offer temporary relief. A complete tuning exercise might be needed to allow the site to handle the complete load on a permanent basis. Also, before restricting site traffic, you should consider other options such as stopping certain marketing and promotional activity.
Throttling at the different layers
Although throttling can be applied to all the layers, it is generally best to start with the layers closest to the shopper:
Content Delivery Networks (CDN) offer services that can be used to throttle load. For example, if you use Akamai, the feature is called SPA (Shopper Prioritization Application). These services allow you to configure rules and conditions to specify which users or requests should go through or wait. CDN is the best place to apply throttling, and this completely prevents requests from reaching the WebSphere Commerce site.
Traffic can also be restricted in the Load Balancers (LB) or Web Servers (IHS). IHS, for example, has settings to specify the maximum number of connections that can be active. These are set in httpd.conf:
<IfModule worker.c> ThreadLimit 25 ServerLimit 64 StartServers 1 MaxClients 600 MinSpareThreads 25 MaxSpareThreads 75 ThreadsPerChild 25 MaxRequestsPerChild 0 </IfModule>
The key configuration in the WebSphere Commerce and Search servers is the Web Container pool. It's size determines the maximum number of requests that can be executed concurrently. If the servers are experiencing 100% CPU, reducing the size of the pool can help.
Keep in mind that a smaller pool doesn't mean less throughput. If the thread pool is over-sized, the overhead of concurrency can hurt performance. Reducing the pool can lead to fewer concurrent threads that complete faster, which in practice, improves throughput. See this post for more details: Key Tuning Configurations: WebContainer Pool.
Databases also have settings to restrict the number of JDBC connections, such as DB2's MAX_CONNECTIONS. As in Commerce the number of database connections is determined by the number of Web Container threads, the database setting is rarely used for throttling.