I Said "Parallelise" Not "Paralyse" Part 1 - Motivation
MartinPacker 11000094DH Visits (6062)
I have enormous trouble pronouncing "parallelise" right - and not saying "paralyse". It's true, and I bet many of you have the same trouble (sober or not). It's on a par with "red lorry yellow lorry" or "the Leith Police dismisseth us".
But it's a word I think we're going to have to get used to pronouncing right. And this post will explain why.
This looks to me like a 4-part series of blog posts on increasing Batch Parallelism. (It started off looking like one but the way it turned into four is perhaps material for another post.)
So, why will parallelising batch become increasingly important? There are really three main reasons:
There is some overlap between these but I think they're sufficiently distinct to draw out separately - which is what the rest of this blog post does.
Increased "Window Challenge"
From a business perspective this is the big one. I'm seeing a number of business trends that are leading to one inexorable conclusion: The delivered growth in speed of "single actors" (batch jobs) will be outstripped - over time - by the need. In other words, you can't long-term just buy yourself out of trouble, whether we're talking about processor speed, disk subsystem or tape speed, transmission line speed, or anything else for that matter.
It's true this varies by installation, and even between applications (or suites) or business lines in an individual organisation. But this is the general pattern. It's also a fact that the pressure comes in waves - because of the nature of the underlying business requirements.
Amongst the business drivers I've seen:
With a single-threaded job stream just one broken application data record can hold up the whole thing. Or the loss of an LPAR or DB2 subsystem or VSAM file.
Partitioned data can mean an increase in resilience. For example:
Taking Advantage Of Capacity
Businesses have tended to size machines by online day requirements and there remains the view that generally it is online that's the peak use of resources. My experience is that about half of installations have batch as the real CPU driver (but probably not the memory driver) and more than half have a bigger I/O bandwidth challenge overnight than during the day.
Where the online day is still the main resource driver an increase in parallelism can usefully absorb the spare capacity overnight.
I contemplate this being a four-part series of blog posts. This part has concentrated on business drivers, almost to the exclusion of technology. The other three posts I expect to be, in order:
The titles and scope might change a little bit as I flesh them out. I'll leave you in suspense as to what "Classification" might be.