Throughput

Throughput is a rate. A rate is the measure of how many actions are completed in a unit of time.

This measure is most often characterized as TPS (transactions per second), TPM (transactions per minute), TPH (transactions per hour), or TPD (transactions per day). Throughput is not the inverse of latency. Your installation might have a high latency of 2 seconds, but a tremendous throughput rate of 2,000 TPS. This throughput might be acceptable in a batch environment where there is not a major concern with individual transaction durations but overall record processing rates.

As an analogy, consider the trip from point A to point B in the Latency example from the highway department perspective. How many cars travel from point A to point B in a day? The highway department concern is not with the latency of an individual car, but with the overall throughput of the highway, measured in cars per day. The department might find that by lowering the speed limit from 70 miles per hour to 50 miles per hour they get more cars from point A to point B in a day (for example, 10,000 versus 12,000). The increase in throughput is because there are fewer accidents that cause overall delays. From an atomic perspective, an individual car is going 29 percent slower, resulting in a longer trip (higher latency), but the overall throughput for the road has risen by 20 percent. Now the highway department might choose to build two more traffic lanes. The overall throughput then doubles from 12,000 to 24,000 cars per day, but the latency is still 3 hours per car. Individual drivers are not going any faster, but a lot more people are now getting from point A to point B in the same time period.

Throughput is typically critical for automated workloads, such as harness-driven inputs or bulk search operations.