Phases of the response time with variability
March 27, 2017  Written by: jlnavarro
Share this post:
Let’s go to the response time versus the number of users signature (graph below) in the simple system described in “Phases of the Response Time”. We made two simplifying assumptions: the service time is constant, and the interarrival time is constant (the the think time is constant). These assumptions allowed us to distill the essence of the response time dependencies, providing us with very useful insights regarding the response time behaviour, the primary parameters it depends on, how it depends on them, and what are its trends and its limits.
Figure 1: The response time versus the number of users for the simple model.
In the real world… variability!
Now let’s go a step further introducing the variability. Customers seldom arrive to a service center at uniform intervals (arrivals side variability). Customers seldom demand the same service time (service side variability). Both magnitudes are essentially variable.
The analysis of the response time with variability is usually done using a powerful mathematical tool called probability analysis. Every time you hit into a queueing theory textbook or article, you’ll find probabilities. Our constant values are transformed in something like this: “it’s twice more probable to have a service time of 2 s than 1 s”, “there’s a probability of 80% that service time lies in the interval (1 s, 2 s)”, “the average response time is 2 s”, and so on. We enter the probabilistic world, where the raw material is random variables, on which we can only state probabilistic facts.
But, for now, I’m interested in highlighting the main consequences of the variability, not in performing an analytical indepth analysis.
Variability Effects
Look at the figures 2 and 3.
Figure 2: No variability case. Uniform arrivals + Uniform service –> No waits –> Best and uniform response time.
This “no variability” figure corresponds to our simple allconstant case (Figure 1). Uniform arrivals and uniform service time result in no waits and the best and uniform response time.
Figure 3: With variability case (arrivals side). Nonuniform arrivals + Uniform service –> Waits –> Worse and variable response time
The “with variability” diagram illustrates the arrivals side variability, in particular batch arrival of users. Nonuniform arrivals result in waits, and the response time seen by users is worse and variable (or volatile).
Ideas to take –> Variability effects are:

An analysis of the same simple model but allowing random exponentially distributed variation of the service time and the think time results in the graph in figure 4.
Figure 4: Average response time for the allconstant (blue) and the random (red) cases..
We can see, for example, that when the user population reaches 80% of the saturation value the average response time for the random model is 4 times the one for the allconstant model, and for 100% of the saturation population the ratio increases to 8 times!
By the way. don’t underestimate the effects of the variability, as it is one of the causes of unacceptable performance from the customer point of view: in general a customer is willing to accept higher but uniform response times than lower but highly variable ones.
Mirror: http://demystperf.blogspot.com/2017/03/phasesofresponsetimewithvariability.html
Climbing the peak
Scenario In your usual sizing efforts you need to know the peak usage% for a certain workload and server(s). What is the right averaging time to capture this peak? Let’s see possible choices. Too long averaging time Averaging Time = 500 min. Peak usage% = 50%. Total loss of finer details. Does this really mean […]
The degraded operations pitfall
Let’s consider an information system supporting a highly critical online activity. Critical means it cannot fail, and if it fails there must be a contingency infrastructure that allows operations to continue “reasonably” well, with zero or tolerable performance impact. Someone trying to reduce acquisition cost decides having half of the processing capacity in the contingency […]
The upgrade sizing pitfall
The art of sizing is not exempt of pitfalls, and you must be aware of them if you want your sizing to be accurate and adequate. Let us talk about a typical scenario: a server upgrade. All the metrics in sizing are measures of throughput, and this has an implication you must take into account: […]
Add CommentNo Comments