Engineering - Part One - A Happy Medium?
MartinPacker 11000094DH Visits (602)
In Engineering - Part Zero I talked about the presentation that Anna Shugol and I have put together. That post described the general sweep of what we’re doing.
This post, however, is more specific. It’s about Vertical Medium logical processors.
To keep it (relatively) simple I’m describing a single processor pool. For example, the zIIP Pool. Everything here can be generalized, though it’s best to treat each processor pool separately.
Also note I use the term “engine” quite a lot. It’s synonymous with processor.
What Is A Vertical Medium?
Before HiperDispatch an LPAR’s weight was distributed evenly across all its online logical processors. So, for a 2-processor LPAR with weights sufficient for 1.2 processors, each logical processor would have 0.6 engines’ worth of weight.
Now let’s turn to HiperDispatch (which is all there is nowadays)1.
The concept of A Processor’s Worth Of Weight is an important one, especially when we’re talking about HiperDispatch. Let’s take a simple example:
Suppose a machine has 10 physical processors and the LPARs’ weights add up to 10002. In this case an engine’s worth of weight is 100.
In that scenario, suppose an LPAR has weight 300 and 4 logical processors. Straightforwardly, the logical processors are:
There are a few “corner cases” with Vertical Mediums, but let me give you a simple case. Suppose the LPAR, still with 4 logical processors, has weight 270. Now we get:
Note that the VM in this case has 70% of an engine’s worth of weight.
How Do Vertical Mediums Behave?
There are two parts to HiperDispatch:
Vertical CPU Management
Let’s take the three types of vertically polarized engines:
The cache effects of the three cases are quite different: It would be reasonable to suppose that a VH would have better cacheing than a VM, which in turn would do better than a VL. I say “reasonable to suppose” as the picture is dynamic and might not always turn out that way.
But you can see that LPAR design - in terms of weights and online processors - is key to cache effectiveness.
We prefer not to run work on VLs - so the notion of parking applies to VLs. This means not directing work to a parked VL. VLs can be parked and unparked to handle varying workload and system conditions.
With Dispatcher Affinity, work is dynamically subdivided into queues for affinity nodes. An affinity node comprises a few logical engines of a given type. Occasionally work is rebalanced.
You could, for queuing purposes, view an LPAR as a collection of smaller units - affinity nodes - though it’s not as simple as that. But that could introduce imbalance, a good motivation for the rebalancing of work I just mentioned.
What Dispatcher Affinity means is that work isn’t necessarily spread across all logical processors.
How Do They Really Behave?
With VMs I have three interesting cases, two of which I have data for. They got me thinking.
So there’s some variability in behaviour. But that’s consistent with every customer environment being different.
Conclusion - Or Should We Avoid Vertical Mediums?
First, in many cases, there’s an inevitability about VMs, particularly for small LPARs or where there are more LPARs than physical engines. I’ll leave it as an exercise for the reader to figure out why every LPAR has to have at least one VH or VM in every pool in which it participates.
I don’t believe it makes any difference in logical placement terms whether a VM has 60% of an engine’s worth of weight or 95%. But I do think a 60% VM is more likely to lose the physical in favour of another LPAR’s logical engine than a 95% VM.
I do think it’s best to take care with the weights to ensure you don’t just miss a logical engine being a VH.
This thinking about Vertical Mediums suggests to me it’s useful to measure utilisation at the engine level - to check for skew. After all you wouldn’t want to have Delay For zIIP just because of skew - when the pool isn’t that busy.
But, of course, LPAR Design is a complex topic. So I would expect to be writing about it some more.