Simultaneous multi-threading

Simultaneous multi-threading is the ability of a single physical processor to simultaneously dispatch instructions from more than one hardware thread context at the same time. Simultaneous multi-threading is designed to produce performance benefits in commercial environments and for workloads that have a high Cycles Per Instruction (CPI) count.

The processor is a superscalar processor that is optimized to read and run instructions in parallel. Simultaneous multi-threading allows you to take advantage of the superscalar nature of the processor by scheduling two applications at the same time on the same processor. No single application can fully saturate the processor. Simultaneous multi-threading also allows instructions from one thread to use all the execution units if the other thread encounters a long latency event. For example, when one of the threads has a cache miss, the second thread can continue to execute. Simultaneous multi-threading is a feature of the POWER5 and POWER6® processors and is available with shared processors.

The following figure shows how simultaneous multi-threading works:

The figure shows single-threaded mode where all physical resources go to a single thread. POWER® systems support single-threading and simultaneous multi-threading.

The figure then shows coarse-grain multi-threading mode where only one thread executes at a time. When a thread encounters a long-latency event, such as a cache miss, the hardware switches to a second thread to use the processing resources, rather than letting the server remain idle. By allowing other work to use what otherwise would be idle processor cycles, this design increases overall system throughput. To conserve resources, both threads share many system resources, such as architectural registers. Therefore, swapping program control from one thread to another requires several cycles.

Finally, the figure shows simultaneous multi-threading mode where the processor retrieves instructions from more than one hardware thread at the same time. The processor schedules instructions for execution from multiple hardware threads concurrently. With simultaneous multi-threading, the system dynamically adjusts to the environment, allowing instructions to execute from each hardware thread if possible, and allowing instructions from one hardware thread to use all the execution units if the other hardware thread encounters a long-latency event.

Simultaneous multi-threading is primarily beneficial in the following contexts:

• In commercial environments where the speed of an individual transaction is not as important as the total number of transactions that are performed. Simultaneous multi-threading is expected to increase the throughput of workloads with large or frequently changing working sets, such as database servers and Web servers.
• Workloads that have a high CPI count. These workloads tend to use processor and memory resources poorly. Large CPI counts are usually caused by high cache-miss rates from a large working set. Large commercial workloads are somewhat dependent upon whether the two hardware threads share instructions or data, or whether the hardware threads are completely distinct. Workloads that share instructions or data, including those that run extensively in the operating system or within a single application, might see increased benefits from simultaneous multi-threading.