Thread your way through the IBM z13 SMT Labyrinth using OMEGAMON XE on z/OS
Laurie072183 1200008KBT Visits (10018)
In the 1 st Quarter of 2015 IBM introduced the IBM z13 processor. The hardware enhancements included Simultaneous Multi-Threading (SMT) of the processor cores. On the IBM z13, using configuration parameters under z/OS Version 2.2, zIIP processors can be configured to run up to 2 simultaneous threads that share a physical core’s resources. Under z/OS each thread is effectively a logical CPU having its own set of related management control blocks and associated metrics.
HiperDispatch introduced z/OS awareness of the topology of the physical cores on a machine and their associated processor cache hierarchies. This enabled intelligent z/OS queueing of workloads’ dispatching units to optimize reuse of instruction and data references from the closest caches to the physical core. Simultaneous Multi-Threading further reduces instruction processing latency at the core level by allowing one thread to access a core’s resources while another thread is forced to wait on resolution of a processor cache miss.
To complement the new hardware, z/OS Version 2.2 (or 2.1, with pre-requisite PTFs) can take advantage of multi-threading on zIIPs by IPLing with a new LOADxx setting, PROCVIEW CORE. Specification of zIIPs to run with 2 threads requires setting a new SYS1.PARMLIB (IEAOPTxx) option (MT_ZIIP_MODE = 2). The MT_ZIIP_MODE setting takes effect at IPL but can also be switched between 1 and 2 threads per core dynamically without an IPL by issuing an operator SET OPT=xx command against an IEAOPTxx member with a different thread value set. Cores configured to run with two threads simultaneously are said to be running in MT-2 mode.
Monitoring cores running in MT-2 mode requires adopting a new performance measurement paradigm that introduces metrics to complement those used for single-threaded processor cores. This blog entry describes how OMEGAMON XE on z/OS Version 5.3.0 Interim Feature 1 (IF1) introduces support for new metrics related to zIIP cores running under SMT.
Internally, the z/OS architecture already supports SMT for all types of CP regardless of characterization. At a yet to be announced date in the future, configuration of multi-threading will be made available for general purpose processors. At that point in time the key differentiator between System z Multi-Threading and other platforms’ exploitation of this technology will become apparent because of System z SMT’s design point of providing consistent charge-back metrics.
New Performance Measurement Paradigm, New Metrics
Before providing examples of OMEGAMON XE on z/OS’s SMT support a familiarity with the new performance metrics associated with SMT is necessary. The following sections describes six key metrics that in addition to existing, single-threaded performance metrics can used for performance measurement in the multi-threaded core environment. OMEGAMON XE on z/OS obtains these metrics using a new z/OS API that returns values at both the Processor Class (CP or zIIP only since CPs can no longer be characterized as zAAPs on an IBM z13) and Logical Core level. These values are calculated using counts of each core’s cycles run and instructions executed when both a single thread is active and multiple threads are active.
Below, screen-shot snippets from the new e3270UI workspace, KM5MT2S, are used to illustrate the OMEGAMON XE on z/OS display metrics described in the following paragraphs.
Multi-Threading (MT) Productivity Percentage
This metric represents the ratio of how much work was actually accomplished to the maximum amount of work that could have been accomplished. OMEGAMON XE on z/OS presents this metric as a percentage to 2 decimal places at both the processor class and logical core level. In single-threaded mode this value is by definition 100.00. A value of 100.00 % reflects all threads on the core were executing work and all core resources are being used. Any value less than 100.00 % indicates the core resources were dispatched to physical hardware but one or more threads on a logical core were in a wait state because they had no work to run.
For a given workload, over any period of time:
Multi-Threading Productivity % = Achieved Capacity Factor / Maximum Capacity Factor * 100
Definitions of the metrics Achieved Capacity Factor and Maximum Capacity Factor follow.
Achieved Capacity Factor
This metric represents the ratio of the actual amount of work a workload accomplished at its current Multi-Threading mode to the amount of work that could have been accomplished by the same workload in single-threaded mode. OMEGAMON XE on z/OS presents this metric to 3 decimal places at the processor class level. In single-threaded mode this value is by definition 1.000. In Multi-Threaded mode, under adequate processor utilization, the range of values varies between approximately 1.000 and 1.400 depending on the Multi-Threading Productivity.
Maximum Capacity Factor
This metric represents the ratio of the maximum amount of work that can be accomplished using all active threads at the current Multi-Threading mode to the amount of work that would have been accomplished by the same workload in single-threaded mode. OMEGAMON XE on z/OS presents this metric to 3 decimal places at the processor class level. In single-threaded mode this value is by definition 1.000. In Multi-Threaded mode, under adequate processor utilization, the range of values varies between approximately 1.000 and 1.400.
The Maximum Capacity Factor typically fluctuates over time depending on the characteristics of the current workload with Batch workloads generally driving the value towards lower values and transactional workloads towards higher values.
On lightly loaded LPARs the value of this metric may vary outside of the expected 1.000 to 1.400 range (actual limits are 0.500 to 2.000) due to the characteristics of the internal processor cycle and instruction count monitoring. A higher value means more capacity is available to workloads running in the processor class than lower values whether the workloads utilize this capacity or not. The previously described metric, Achieved Capacity Factor, measures how much of this available core capacity is actually used by all threads running on the processor class’s cores.
Average Thread Density
This metric represents the average number of a processor class’s cores’ active threads that were dispatched to the hardware’s physical cores. OMEGAMON XE on z/OS presents this metric to 3 decimal places. In single-threaded mode this value is by definition 1.000. In Multi-Threaded mode the range of values varies between 1.000 and 2.000.
Core Busy Time
This metric is the amount of time (in milliseconds) a logical core was dispatched to a physical core over some interval. OMEGAMON XE on z/OS is required to use the new z/OS API to obtain this value for use in the LPAR Busy Percentage utilization calculation for dedicated processors. OMEGAMON XE on z/OS presents the LPAR Busy Percentage metric to 2 decimal place at both the processor class and logical core level.
Multi-Threading Utilization Percentage
This metric represents the percentage of the maximum core capacity that was used over a period of time. OMEGAMON XE on z/OS presents this metric as a percentage to 2 decimal places at both the processor class and logical core level.
For a given workload, over any period of time:
Multi-Threading Utilization % = Multi-Threading Productivity % * LPAR Busy Time
New e3270UI Multi-Threading Details Workspace
Multi-Threading support in OMEGAMON XE on z/OS introduces a new workspace, KM5MT2S “Multi-Threading Details” that displays the previously described metrics in addition to existing, complementary metrics that are also available in the workspace, KM5HDDS “HiperDispatch Details”. The HiperDispatch Details workspace has been modified to display metrics by thread at the logical CPU level in Multi-Threaded mode.
Navigation can be performed to the Multi-Threading Details workspace from an e3270UI session by entering option “z” in the Sysplex Name column input field of the “All Active Sysplexes “sub-panel of the “Enterprise Summary” workspace, KOBSTART.
Then select option 4 or “D” from the z/OS System Resources pop-up to navigate to the “System CPU Utilization” workspace, KM5CPUS.
Enter option “M” in the LPAR Group Capacity Limit column input field of the “LPAR Utilization” sub-panel to navigate to the “Multi-Threading details” workspace, KM5MT2S. Once in KM5MT2S the SMF ID input field in the top right corner of the workspace may be used to navigate between different LPARs’ Multi-Threading Details metrics.
LPAR Cores Summary sub-panel
The “LPAR Cores Summary” sub-panel metrics, shown below, displays the number of logical cores configured to the LPAR, the number of threads running across all of these cores and the measurement interval, in seconds to 3 decimal places that the workspace metrics represent.
LPAR Processors Summary sub-panel
The “LPAR Processors Summary” sub-panel metrics, shown below, displays by Processor Class (CP or zIIP) the previously described metrics. In addition the Multi-Threading Mode (1 = single-threaded, 2 = 2 threads running on each logical core), LPAR Busy percentage, across all logical cores, and MVS Busy Percentage, across all of logical cores’ threads in a processor class are displayed.
Multi-Threading Details sub-panel
The “Multi-Threading Details” sub-panel metrics, shown below, displays by Logical Core ID the previously described metrics. In addition the Core Type (CP or zIIP), LPAR Busy Percentage, HiperDispatch Priority, HiperDispatch Share Percentage, Core Online Percentage are displayed for each logical core and only appear on rows related to Thread ID 0. All previously mentioned metrics apply to the entire logical core but the MVS Busy Percentage and Parked Percentage apply to each thread running on a logical core.
The MVS Busy Percentage is of interest in Multi-Threading mode because each thread is effectively treated as a logical CPU by z/OS. In the above screen-shot the MVS Busy Percentage values across threads 0 and 1 are 74.00% and 73.51% respectively. While the MVS Busy Percentage is a well-known metric in single-threaded mode, in multi-threaded mode the multiple values actually present a workload’s view from within z/OS of multiple logical CPUs utilization that are running on a single logical core.
New OMEGAMON Classic 3270 MSMT Command
A new MSMT immediate command is introduced in the OMEGAMON 3270 interface that provides all of the metrics previously described for the OMEGAMON XE on z/OS e3270UI. The OMEGAMON 3270 screen-shot below shows the format of this support.
Modified OMEGAMON Classic 3270 HDSP and SYS/ENV Commands
The existing HDSP command displaying HiperDispatch metrics has been modified to include the thread ID associated with the detailed metrics. In the screen-shot below notice that the zIIP processor section displays two rows of metrics for each LCPU ID under the single character “T” sub-header.
The SYS Major command’s ENV Minor command has been enhanced to display the current multi-threading status and configuration information for the LPAR such as the maximum number of threads per core and the number of threads configured for the CP and zIIP processor classes. Notice the new information in the last two rows of the display output.
This blog entry has described the support for the IBM z13’s Simultaneous Multi-Threading feature added in OMEGAMON XE on z/OS Version 530 Interim Feature 1. Hopefully, this blog entry has provided an informative introduction to the IBM z13’s Simultaneous Multi-Threading feature and OMEGAMON XE on z/OS support for the new performance measurement metrics. As experience is gained with the characteristics of SMT, and how to optimize its use, additional blogs will be posted.
SMT support is available via the following links:
OMEGAMON XE on z/OS Version 5.3.0 PTF UA76496: http
OMEGAMON XE on z/OS Version 5.3.0 Interim Feature 1: http