The growth of e-business makes it critical to ensure that the IT infrastructure supporting the Web sites can provide available, scalable, fast, and efficient access to the company's information, products, and services. More than ever, CIOs and their teams struggle with the challenges to minimize downtime and network bottlenecks and maximize the use of the hardware and software that comprises their e-business infrastructure.

IT infrastructure supporting most high-volume Web sites (HVWSs) typically consists of multiple layers of machines, frequently called tiers, as illustrated in Figure 1. Each tier consists of multiple machines, from two to hundreds, to provide capacity and availability for the functions running at that tier. A tier provides a particular set of functions, such as serving content (Web presentation servers), providing integration business logic (Web application servers), or processing database transactions (transaction and database servers).

Figure 1. Multi-tier infrastructure for e-business

Even with this growing complexity, it's possible to analyze typical IT infrastructures and develop related models to assist in predicting and planning how to meet future requirements.

A methodology for capacity planning

IBM's IT experts have been working with many IBM customers to analyze large Web sites and help customers implement scalable Web sites (see Resources number 1). Some of these customers are already working with IBM to exploit and further the technologies for capacity planning being developed. Our methodology is based on this ongoing research (Resources numbers 4, 5, 6, 7, 8, 9), as well as IBM's patterns for e-business.

Our methodology for capacity planning is based on our analysis of many large Web sites, including IBM's, and on our continuing engagements with large customers seeking to improve site performance, accurately project workloads, and make infrastructure changes that will satisfy future requirements. The methodology consists of four steps:

1. Identify your workload pattern.
2. Measure performance of current site.
3. Analyze trends and set performance objectives.
4. Model your infrastructure alternatives.

These steps work equally well whether you are considering changes to a current site or planning a new site.

This paper focuses on technologies for capacity planning. Implementing such technologies will affect your IT organizations, processes, and people; the paper does not address those related implications.

Our team's methodology for capacity planning complements IBM's patterns for e-business (see Resources), in that you can map each workload pattern we describe to one of the e-business patterns for site design. Regardless of the methodology you used to design your site, this capacity planning methodology can complement that effort and establish a foundation for managing your capacity requirements.

Step 1. Identify your workload pattern
Because capacity planning is an issue mainly for high-volume Web sites, for this paper we assume that your workload is high-volume and growing, serving dynamic data, and processing transactions. Beyond that, you must consider other characteristics, such as transaction complexity, data volatility, and security. After your analysis, you can fit your workload pattern into one of five classifications: publish/subscribe, online shopping, customer self-service, trading, or business-to-business. Correctly identifying your workload pattern assures the best results from the rest of the steps in our methodology and maximizes your site's chances for satisfying future requirements. (If you have been following our team's series of papers, you may not need to review the workload pattern and so may prefer to skip ahead to Step 2.)

Consult these workload-pattern descriptions (or see Workload patterns at a glance for the essence, distilled into a table) to identify your workload pattern:

• Publish/subscribe Web sites provide users with information. Sample publish/subscribe sites include search engines, media sites such as newspapers and magazines, and event sites such as those for the Olympics and for the championships at Wimbledon. Site content changes frequently, driving changes to page layouts. While search traffic is low volume, the number of unique items sought is high, which is why this type of site generates the largest number of page views of all site types. As an example, the Sydney Olympics site (a WebSphere environment) successfully handled a peak volume of 1.2 million hits per minute (see Resources for WebSphere info). Security considerations are minor compared to other site types. Data volatility is low. This site type processes the fewest transactions and has little or no connection to any legacy systems.
• Online shopping sites let users browse and buy. Sample sites include typical retail sites where users buy books, clothes, and even cars. Site content may be relatively static -- such as a parts catalog -- or dynamic, where items are frequently added and deleted as, for example, promotions and special discounts come and go. Search traffic is heavier than the publish/subscribe site, though the number of unique items sought is not as large. Data volatility is low. Transaction traffic is moderate to high, and almost always grows. The typical daily volumes for many large retail customers range from fewer than one million hits per day to more than 13 million hits per day, and with a range from 100,000 transactions per day to 3,000,000 transactions per day in the top range; of the total transactions, typically between 1% and 5% are buy transactions. When users buy, security requirements become significant and include privacy, nonrepudiation, integrity, authentication, and regulations. Shopping sites have more connections to legacy systems, such as fulfillment systems, than the publish/subscribe sites, but generally fewer than the other site types.
• Customer self-service sites let users help themselves. Sample sites include banking from home, tracking packages, and making travel arrangements. Data comes largely from legacy applications and often comes from multiple sources, thereby exposing data consistency. Security considerations are significant for home banking and purchasing travel services, less so for other uses. Search traffic is low volume; transaction traffic is low to moderate, but growing.
• Trading sites let users buy and sell. Of all site types, trading sites have the most volatile content, the highest transaction volumes (with significant swing), the most complex transactions. Trading sites also are extremely time-sensitive. Trading sites are tightly connected to the legacy systems, using software like IBM's MQSeries for connectivity. Nearly all transactions interact with the back-end servers. Security considerations are high, equivalent to online shopping, with an even larger number of secure pages. Search traffic is low volume.
• Business-to-business sites let businesses buy from and sell to each other. Data comes largely from legacy applications and often comes from multiple sources, thereby exposing data consistency. Security requirements are equivalent to online shopping. Transaction volume is moderate, but growing; transactions are typically complex, connecting multiple suppliers and distributors. There are two styles of this pattern:
  • Business-to-business integration: This style includes programmatic links between arms-length businesses (where a trading partner agreement might be appropriate). Supply chain management is an example.
  • e-Marketplace or B2M2B: The M represents the e-Marketplace, which supports multiple buyers and suppliers. The buying function can be performed online or programmatically.

Step 2. Measure performance of the current site
As always, you must understand the present before you can plan the future. You need to measure these site characteristics: volumes (hits, page views, transactions, searches), arrival rates, response times by class, user session time, number of concurrent users, and processor and disk utilization. If you're planning a new site, you estimate the metrics by referring to your related experience compiled in site profiles, or you can base your estimates on the site profiles of your experienced consultants, such as our HVWS team.

Our analysis of the performance of e-business infrastructures under various workload patterns demonstrates that workload pattern complexities (for example, bursty arrival patterns) can significantly affect resource demands, throughput, and the latency encountered by user requests, in terms of higher average response times and higher response time variance. Without adaptive, optimal management and control of resources, service level agreements (SLAs) based on response time are impossible. The capacity requirements on the site are increased while its ability to provide acceptable levels of performance and availability diminishes.

When analyzing your current site, remember to take into account the design of your Web pages. IBM's work suggests that there are many practices you can follow to reduce the time your Web pages take to download. Web pages have common components and characteristics, such as page size and number of items, that you can and should manage with an eye toward minimizing download time. Doing the "right" thing will not always be possible, and some components or characteristics may be outside of the control of the page designer. Still, everyone with an interest in the site's performance should understand these factors and their related tradeoffs. Table 1 summarizes page design metrics from 15 different Web sites. The metrics vary but strongly suggest that page design is an important performance component; when managed well they can improve a site's capacity. In the color-coded table, metrics considered good or excellent are green; less favorable metrics are amber, and unacceptable metrics are red. For details on optimizing your pages for quick downloading, read one of our team's previous papers, "Design pages for performance."

Table 1. Example Web page metrics. Green means good; red means unacceptable; amber means marginal.

Back to top

Understand workload metrics

Correctly identifying your workload pattern prepares you for measuring and understanding the complexities of your site. Each workload pattern has an associated class of user requests. Figure 2 shows an example of classes of user requests associated with the online shopping workload pattern.

Each class is characterized by how the requests arrive at the Web site and the resources required to satisfy the request. The major factors that affect arrivals include standard (marginal) distribution, dependence structure, and seasonality. In general, IBM's analysis demonstrates complex behaviors that include light-tailed and heavy-tailed distributions, short-range and long-range dependencies, strong seasonality and periodicity, and geographic effects. The typical assumption of independent exponential interarrivals of Web requests does not hold true under these conditions and one has to solve the problem with nontraditional assumptions that require complex mathematical algorithms. IBM's mathematical research of these characteristics has enabled the development of improved models to understand and predict the impacts of these relationships and behaviors.

Figure 2. Each workload pattern has an associated class of user requests

Distribution and dependence
Web traffic exhibits bursty, heavy-tailed, and correlated arrival patterns. Bursts refer to the random arrival of requests, with peak rates far exceeding the average rates. These bursts are caused by unpredictable events such as major stock market swings or special events such as Christmas or Valentine's day. Such events yield dependencies among requests (for example, larger bursts tend to occur in close proximity), heavy-tail distributions (for example, very high variability in the sizes of the bursts), and the combination of dependencies and heavy-tail distributions. A heavy-tailed distribution for a random variable is one where the tail of the distribution decreases subexponentially. For these distributions, a large value might occur. The batch arrival process exhibits such heavy-tailed behavior and the batch request sizes tend to be strongly correlated. A practical consequence of burstiness and heavy-tailed and correlated arrivals is difficulty in capacity planning.

Burstiness and wide-ranging hit rates are among the most obvious workload pattern complexities that affect Web site performance and availability. In traditional models, requests are independent and the variance in the burst sizes are relatively small. These distributions belong to the class of light-tailed distributions. The burstiness of a HVWS yields heavy-tailed distributions and a strong dependence structure. This is illustrated by the traffic patterns from the 1998 Nagano Olympic games, as shown in Figure 3. Such bursts of requests are triggered by some special event, for example, in this case when Japan won the Gold medal for ski jumping in Nagano. Contrast the heavy-tailed distribution from Asia with the light-tailed distribution on the same day from Europe. Further, note the dependence structure from day to day, as well as within each day, at both locations.

Figure 3. Traffic patterns from 1998 Nagano Olympic Games

IBM's Wimbledon 2000 Web site also exhibited extreme bursts on its busiest day, 7 July 2000.
Figure 4 graphs the record-breaking site traffic on that day, when peak hits per minute reached 963,948 and peak hits per day totaled 281,605,872.

Figure 4. IBM's Wimbledon Web site on its record-breaking day

Nontraditional request traffic stresses the Web server. Bursty traffic with heavy-tailed distributions degrades the performance by several orders of magnitude over light-tailed distributions. For heavy-tailed distributions, the extremely large bursts occur relatively more frequently than the light-tailed model. Moreover, the dependence structure causes these bursts to occur in close proximity to each other. With such input traffic characteristics, the performance metrics, in particular, the response time, have similar characteristics as the input traffic. This helps to explain why some sports and e-business Web sites are more difficult to maintain than relatively simple Web sites (for example, a university Web site serving only static content).

With respect to SLAs, the same level of service for heavy-tailed distributions requires a more powerful set of servers, compared with the case of independent light-tailed request traffic. To guarantee good performance, you need to focus on peak traffic duration because it is the huge bursts of requests that most degrade performance. That is why some busy sites require more head room (spare capacity) to handle the volumes; for example, a high-volume online trading site reserves spare capacity with a ratio of three to one.

Seasonality
Seasonality refers to the periodicity of the request patterns. Seasonal traffic is most often represented by the regular daily activities of the users of a Web site. For example, traffic to some e-trading Web sites has consistent peaks and valleys each day when the market opens and closes. Seasonal traffic is also observed in monthly intervals, for example, when users pay bills at the end of the month, and during designated periods, for example, the holiday season.

Figure 5 shows an example of seasonal traffic from the Nagano Olympics Web site. The figure plots the number of requests received every five minutes by all servers from February 9 through February 16. While each daily cycle varies considerably, note that each day has three peaks and that overall traffic intensity increases each weekday then decreases on the weekend. These patterns repeated each week, demonstrating seasonal variations that correspond to weekly cycles.

Figure 5. Example of seasonality, demonstrated by one week from Nagano Olympics traffic

Seasonal requests can degrade the performance of the Web server because, for the peak duration, large batches of requests occur around the same time. The central questions are how high is the peak and how long is the peak duration. The answers to these two questions can have a significant impact on how powerful the Web server should be in order to handle a specific SLA. To satisfactorily handle request traffic, the capacity of the Web server should be close to peak request level, with some head room to allow for unexpected growth.

Other factors that define the workload pattern include the volume of page views and transactions, the volume and type of searches, the complexity of transactions, the volatility of the data, and security considerations.

The rest of this section introduces techniques available to obtain the measurements you need to complete your capacity plan.

Back to top

Obtain site measurements

Each workload pattern requires specific measurements. Table 2 gives an example of some current measurements of an online shopping site, the "today" measurements to use as a baseline for projections.

Table 2. Example of baseline measurements of an online shopping site

By analyzing typical user visits, it's possible to create probabilities about future user visits. Online shoppers, for example, typically browse, may search, and occasionally buy. You can develop various scripts to describe user visits. Scripts 1, 2, and 3 contain samples of scripts for online shopping, online banking, and online trading scenarios that you can adapt for your circumstances.

Script 1. Example script for online shopping scenario

Script 2. Example script for online banking scenario

Script 3. Example script for online trading scenario

Using the scenario scripts and the data from your measurements, you can create what is called a transition matrix. Figure 6 is an example of a transition matrix for an online shopping visit. Viewing the sample transition matrix as it relates to the sample script, you can easily see the browse and search requests; the buy request occurs when the user decides to add (to shopping bag) and then pay.

Figure 6. Example of a transition matrix for an online shopping visit

Step 3. Analyze trends and set performance objectives
Your workload is growing and your current metrics, no matter how good they are, must improve -- along with the capacities of the hardware and software that constitute your infrastructure. In this step, you analyze trends to determine the characteristics of future peak volumes and then set objectives for each metric identified in Step 2, along with any new metric that applies to your future requirements. Table 3 gives an example of current and projected measurements for an online shopping site. Performance objectives are usually driven by business objectives, for example: to improve response time to preferred customers.

Table 3. Example of current and projected measurements for an online shopping site

The ability to accurately forecast request patterns is an important requirement of capacity planning. Our forecasting methodology is based in part on constructing a set of mathematical methods and models that isolate and characterize the trends, interdependencies, seasonal effects, randomness, and other key behavior in the Web site's request patterns (see Resources numbers 4, 6, 7, and 8). This includes, for example, the use of piece-wise autoregressive moving average (ARIMA) processes combined with heavy-tailed distributions. Figure 7 is an example of a request pattern we would consider for our models; it shows the number of requests received per second over the period of one hour at the Nagano Olympic site. The curve fitted to these measurements is shown in the middle of the data points.

Since each of the key traffic characteristics can scale differently, we use a general set of mathematical methods to estimate the intensity of each characteristic in future request patterns and to scale each characteristic to its forecasted intensity (see Resources numbers 4, 7, and 8). We then combine these scaled mathematical models to characterize and predict request patterns. By using our mathematical methods to isolate, characterize, and forecast the trends, interdependencies, seasonal effects, randomness, and other key behavior in the request patterns, we have developed a general methodology that provides a more accurate and effective approach for predicting request patterns than other approaches being used today.

Figure 7. Requests per second over one hour during Nagano Olympics

Our methodology has proven effective in practice, having been used to predict request patterns of actual sites over both short-term and long-term time frames. In particular, we used our methodology to predict the peak hour request volumes for a recent sporting-event Web site hosted by IBM. We based estimates on request patterns from the Web site in three previous years, together with 95% confidence intervals. Among other approaches, we used a first-order rate-of-change method for estimating the traffic intensity scaling factor from year to year. Our comparisons for 1997, 1998, and 1999 revealed a trend for exponential growth. We then used the forecasted scaling factor for 2000 with our methodology to scale the peak-hour traffic model from 1999 to forecast a peak-hour traffic model for 2000. And it worked as predicted: our forecasts agreed closely with the site's actual peak volumes. Moreover, these methods have been applied with equal success to estimate request patterns for upcoming seasonal events, such as the Christmas online shopping rush. We also have used our methodology to forecast request patterns over short-term time frames (days, weeks); again, our forecasts were found to be in excellent agreement with the site's actual request patterns.

Step 4. Model your infrastructure alternatives
You are ready to determine the components needed to construct your site's infrastructure. Now you match components to the particular requirements and objectives of your workload pattern (with the aid of your consultants or solution engineers from your friendly global vendor, perhaps).

The technologies IBM is developing for HVWS capacity planning rely on the three models depicted in Figure 8.

Figure 8. Models for HVWS capacity planning

The business model, or business usage model, defines the e-business pattern and workload pattern. There is a user behavior model for each workload pattern. Each workload pattern consists of several classes of user requests. The arrival patterns and routes (transition matrices) site visitors follow for each class comprise the user behavior model. The hardware and software resources, and the amount of each required to satisfy each class of user requests, comprise the infrastructure model.

The infrastructure model processes browse, search, and buy transactions. The model assumes:

• The Web site has multiple layers of machines, or tiers, each handling a particular set of functions, such as the site depicted in Figure 9.
• A load-balancer, such as the network dispatcher, routes requests to multiple Web front-end servers using an algorithm to distribute requests evenly among the servers.
• The front-end Web servers handle requests for static or dynamic pages.
• The Web application server processes business logic for the transactions initiated by the request. In Figure 9, the account and quote servers are the application servers.
• The back-end database server handles requests for dynamic pages that involve obtaining or updating information; such requests do not return through the load balancer.

Figure 9. A Web site with multiple tiers (not including firewall layers)

We formulate a class of queuing networks to model multi-tier architectures in order to analyze performance at different levels. We further derive a variety of solutions to these models under different input traffic patterns and at different time-scales. This family of mathematical models and solutions are general enough to abstract the underlying hardware and software details but detailed enough to produce meaningful performance results. We consider the queuing system, where the resources of each tier is modeled by multiserver queues that have specific relationships to one another. These relationships are determined through measured or estimated workload characteristics. We then solve the performance/capacity problem against a set of user requirements, such as the number of concurrent users, response time, or throughput rate. We have also developed unique formulas to allow us to estimate the behavior of the system where peak demands are significantly higher than average demand, and there is a non-negligible probability of accessing large amounts of data by the user.

Our method is flexible enough to model horizontal and vertical scaling, or a combination, depending on user requirements and workload characteristics. For example, we can increase the number of Web servers by adding another server or by adding another processor on a given server. Given the appropriate Web site and workload data, we are then able to obtain performance estimates from our performance models and analysis.

We have developed capacity plans for a number of IBM-hosted Web sites. Figure 10 depicts one such site and reflects the process of calibrating our model using current data from the site, then developing projections based on current data, trends, and objectives. In Figure 10, the first three response time curves reflect the current data used to calibrate the model as discussed in Step 2. By analyzing current metrics and component information, we are able to project the fourth curve.

Referring to Figure 10, the results show that when the request traffic is light, one front-end server is enough to handle the load. As the traffic increases, the response time curve remains flat until the front-end CPU reaches a utilization of 90% (2.8 million hits per hour). At this point, a minor increase in the load can rapidly plunge the system into a situation resembling deadlock, where the front-end server attempts to serve more and more files at slower and slower speeds, such that few are experiencing satisfactory response times. This means that the front-end server has become the bottleneck. We therefore need to add a front-end server, and upon doing so, the front-end CPU utilization drops as desired. The response time curve becomes flat until the front-end CPU again reaches a utilization of 90% (5.1 million hits per hour), when we need to consider adding another front-end server. The back-end server becomes the bottleneck only after about 15 front-end servers are handling around 28 million hits per hour.

Figure 10. Scaling a WebSphere Commerce Web site

The graph in Figure 11 is a sample of one that we produce when we analyze performance objectives against specific hardware and software components.

Figure 11. Sample graph showing components of performance.

Back to top

Summary

The challenge of effective capacity planning for high-volume Web sites is an awesome one, but it's not insurmountable. The methodology we suggest in this paper offers a road map toward understanding your workload pattern and current metrics, analyzing trends and setting objectives for the future, and, finally, selecting the IT infrastructure components needed to meet your performance objectives. The ability to analyze your site's requirements in the context of your particular workload pattern can contribute greatly to making the right selections.

Of course, the subject of capacity planning is an ongoing study. Increasingly valuable information is available (see Resources number 10), exciting new tools are emerging, and capacity-on-demand options will also greatly enhance the ability to respond to traffic growth. With an eye toward discovering and documenting modern design practices that allow ever greater capacities and scalability, IBM's HVWS group will continue refining its methodology, developing tools that embody that methodology, fine-tuning its mathematical methods, and looking at the additional challenges presented by such areas as network caching and the fast-growing business-to-business workloads. The promise remains great, however, of meeting future needs while planning effectively and reducing costs.

Printed reference resources

Workload patterns at a glance

Resources

• A.K. Iyengar, M.S. Squillante, L. Zhang. Analysis and characterization of large-scale Web server access patterns and performance. World Wide Web, Vol. 2, 1999.
  
  
• W.N. Mills, M.S. Squillante, C.H. Xia, Li. Zhang. Web workload service requirement analysis: A queueing network approach. IBM Research Report, 2000.
  
  
• Z. Liu, N. Niclausse, C. Jalpa-Villanueva. Web traffic modeling and performance comparison between HTTP1.0 and HTTP1.1. System Performance Evaluation: Methodologies and Implications, pp. 177-189, 2000.
  
  
• M.S. Squillante, D.D. Yao, L. Zhang. Web traffic modeling and Web server performance analysis. IEEE Conference on Decision and Control, December 1999.
  
  
• D. A. Menasce and V. A. F. Almeida. Capacity Planning for Web Performance: Metrics, Models, and Methods, Prentice Hall, 1998.
  
  
• D.A. Menasce and V.A.F. Almeida. Scaling for E-Business: Technologies, Models, Performance, and Capacity Planning, Prentice Hall, 2000.
  
  
• Find out about the IBM software that ran the Sydney Olympics site: IBM's WebSphere Application Server, WebSphere Commerce Suite, and MQSeries.
  
  
• Find out about the IBM WebSphere Commerce Suite, used by customers who run large-scale online shopping sites that we have studied.
  
  
• Find out more about the software used by trading sites we've studied: WebSphere's Application Server and MQSeries.
  
  
• Find out about more IBM's Capacity Advantage tool.
  
  
• Design for Scalability, December 1999
  
  
• Web Site Personalization, February 2000
  
  
• Design Pages for Performance, May 2000
  
  
• The Patterns for e-business site helps you get the big picture for planning a complex site architecture and provides decision support for the process.
  
  

About the author

Comments

Back to top

Trademarks  |  My developerWorks terms and conditions

Table of contents

• A methodology for capacity planning
• Understand workload metrics
• Obtain site measurements
• Summary
• Resources
• About the author
• Comments

Local resources

• IBM Innovation Center - Austin, TX
• IBM Innovation Center - Chicago, IL
• IBM Innovation Center - Dallas, TX
• IBM Innovation Center - San Mateo, CA
• IBM Innovation Center - Waltham, MA
• Technical Briefings in North America

My developerWorks community

Dig deeper into WebSphere on developerWorks

Rate a product. Write a review.

Write a review for Rational products today!

Special offers