Martin Packer has been a member of IBM's global mainframe Performance Consulting team for many years and has contributed strongly to IBM's performance analysis tools for z/OS, DB2 and Batch tuning. He presents at major z/OS customer conferences in the USA and Europe and has authored a number of IBM Red Books. He has good links to IBM's product developers and to a number of well-known external authorities on mainframe performance.

Latterly he's got into web programming, around Firefox. This has included doing stuff with GreaseMonkey and, latterly, writing a Firefox extension.

System z Technical Conference - Dresden

I'm going to try something different this year – and composing this using OpenOffice Writer on my new ASUS EEE PC while flying home is certainly different. But what's new is not documenting all the bits and pieces I learnt. Instead I'm going to give some impressions:

• On Monday afternoon we had an enforced evacuation from the conference centre and much of the environs – because of a World War II bomb (that the local paper said was dropped by the US Air Force.) I think everybody was forced to tour Dresden  - which is well worth visiting. It's a great cultural city.
• Next time I'm going to ditch the jacket as it made me think I had to drag my backpack on wheels one mile each way each day. Far better to carry it – except in San Antonio last year when it was far too hot and humid.
• Next time I'm going to follow the conference instructions and book my hotel well in advance so I don't have to do the long commute on foot each day. Yeah right.
• Next time I'm going to be fit enough to go the distance. Yeah right.
• It's handy to sit in sessions you already know a fair amount about – to get you thinking. Example: Peter Enrico's excellent presentation on WLM samples. (I think my code needs working on as a result of that.)
• There is lots going on in the “Web 2.0 and z/OS” space. Thanks to Kevin Keller and Holger Wunderlich for that.
• Perhaps I shouldn't assume everyone knows what CFCC Dynamic Dispatch is. I got a question on that.
• There might be “unintended consequences” in z10 HiperDispatch disabling IRD's Logical CP Management. And there was a good question on that.
• Colin Paice from Hursley really is a very good speaker.
• Twitter is useful as a way of raising consciousness about conferences going on -and it's also good at helping people find each other.
• It was good to run into Aneel Lakhani in person. He's in the category of “I don't know precisely why I know him but I know him from BlogCentral and I'm glad I know him” people. (BlogCentral is IBM's internal blogging site.)
• To the customer who said she was doing a thesis on Web 2.0 yes we really do encourage all IBMers to dive head first into Web 2.0 and social networking in general. Yes, we have guidelines (which I contributed a little to) but nobody finds them restrictive. I wish more companies were like this.
• It's great to have a tour guide who grew up in this part of the world and who went to university in Dresden. Thanks Barbara!
• My manager is following me on Twitter. But that knowledge doesnt seem to have inhibited me in the slightest. I'm still goading him into (hopefully) joining me there.
• I got a chance to forewarn lots of Poughkeepsie folks I'll be in town in a couple of weeks. If they scarper, given this amount of notice, I will take it personally.
• I found lots of customers considering doing Data Sharing at non-trivial distances. So the Redbook ought to reflect that.
• People agreed with me that the CF Structure CPU support in CFLEVEL 15 and RMF will make life much easier for Parallel Sysplex customers.

So, roll on UKCMG in only a few days time. When I won't have to watch my language so carefully.

And my experience of composing this on the EEE is:

• The machine is nice and small – so it doesn't feel cramped on the tray table.
• The keyboard will take some getting used to as it's small (but the screen, though small, is just fine).
• OpenOffice is a new package to me so I'm finding its quirks. (But I have to find them somehow anyway.)
• OpenOffice's “Export to HTML” seems to export in a fairly style-rich way when I'd rather have had minimalistic HTML. I expect I can easily fix that.
• OpenOffice's “Export to PDF” works really nicely (and quickly). I was only messing about when I tried it but I was impressed.

I really do like this EEE. It's a cheap Linux machine that's easy to do LAMP stuff on. (On the way to Dresden I learnt a bit of Ruby.) But this has been the first (and I think successful) test of using it to do real work. If you can call blogging real work.

And the motivation for trying out Linux and OpenOffice is because I expect to be using them on my main machine within the next month or two. I have to start on the learning curve somewhere (or at least some time).

And finally thanks to all the customers and fellow presenters who made this a great conference for me.

Misunderstood Coupling Facility Commands

I wonder how many of you recognise the terms "READ_COCLASS", "WARM", "RFCOM","WRITE_DATALIST" and "CASTOUT_DATALIST".

Granted that in that list are two pairs of terms that I believe to be synonymous, but all five are relatively recent enhancements to Parallel Sysplex infrastructure that have a positive impact on DB2 Data Sharing performance (or at least the management of Performance).

As I mentioned in a previous post I'm going to Poughkeepsie soon to work on a Parallel Sysplex Performance Redbook - which will deal with topics such as "Data Sharing at a Distance" and "New Parallel Sysplex Technologies". In preparation for this (not that I do much preparation for such things) I read through the various sections on Data Sharing Performance in the various DB2 Performance Topics Redbooks, starting at the beginning - with Version 4.

It might be significant that I don't really understand these Coupling Facility commands. But then again it might not be... given that there are (without being self-deprecating) far better proponents of DB2 Data Sharing performance. In any case I feel the material that's out there falls short in terms of describing when these commands are used and what benefit they bring (and what the downsides, if any, might be. So I hope to write some material in this area and ask (what I hope to be) searching questions of Development.

Now, these commands were first exploited by DB2 Versions 6 and 8. So the material might end up on the cutting-room floor. After all we're all interested in System z10, CFLEVEL 15 and DB2 Version 9. Right? What I'm hoping to do - in any case - is to share some of that material through this blog. If I succeed in doing that I hope you'll help me by feeding back on the blog entries. Corrections, additional questions, etc.

And the fun really begins on May 22nd. Just after UKCMG.

Categories : [   DB2  |  coupling  |  facility  |  parallel  |  sysplex  ]

May 03 2008, 01:43:38 AM EDT Permalink

System z Technical Conference - Dresden

This is a great conference and I always enjoy attending it (and presenting at it). Talking of which, I have three sessions this year:

• Much Ado About CPU which has a fair modicum of new stuff, including material on System z10.
• Memory Matters in 2008 for which the updates are mainly in the System memory area, particularly the 1MB Pages support in System z10
.
• DB2 Data Sharing Performance For Beginners which actually isn't much changed. (As I will be spending late May and much of June in Poughkeepsie working on a Parallel Sysplex Performance Redbook maybe the next iteration will be substantially changed.)

I'm also looking forward to a number of other sessions. No doubt I'll get to say a few words in those as well. In particular Kevin Keller's session on "Web 2.0 and z/OS" has caught my fancy. (As you probably know I really do feel we in the mainframe community ought to pay attention to things like Web 2.0 and bid for z/OS to run the infrastructure.)

So I hope to see lots of you in Dresden.

Categories : [   memory  |  z/OS  |  z10  ]

May 03 2008, 01:31:30 AM EDT Permalink

My HackDay 5 Project - z/OS System Logger Analysis

New for HackDay5 was the "zHackDay" initiative - which I take "responsibility" for...

The idea was that mainframe-related hacks were labelled as such. Partly to goad mainframe folks into participating, and partly to point out to non-mainframers that we're just as capable of doing fun hacks as everyone else.

So, was my project exciting?

Probably not - but then I wasn't personally aiming to set the world alight. I'm more interested in getting useful stuff done that I wouldn't ordinarily get around to. So my project is basically to map the SMF 88 System Logger records.

This has been done before - and indeed there's a z/OS-supplied utility to report on System Logger from Type 88 data.

So why would I "reinvent the wheel"?

The main reason is that it allows me to put the data together with other things. Such as SMF 74 Subtype 4 Coupling Facility data. If a logstream uses a coupling facility structure the SMF 88 record tells you the structure's name. (This is similar to a recent trick where the 74-2 XCF data tells you the structure name and my code looks up the 74-4 view of the structure.)

Actually I know relatively little about System Logger - but I'm learning fast. And that's the other reason I did this hack... To enable me to learn.

Finally, if I discover something's missing from the data I can use real customer situations to discuss with Development what's needed.

So, how far did I actually get? I got the Type 88 Subtype 1 (Logstream Activity) data mapped and summarised at the (e.g.) hourly level. And some very simple reporting. I haven't (yet) done the same for the Type 88 Subtype 11 (Coupling Facility Structure) data. But that should be easy and straightforward.

So, if you're sending me data I'm going to be asking you to send me your Type 88 data as well. I hope you'll find what I do with it to be useful.

Categories : [   88  |  hackday  |  hackday5  |  logger  |  z/OS  |  zhackday  ]

Apr 27 2008, 08:59:14 AM EDT Permalink

Moron SPAM :-)

• Delete SPAM rather than mark it invisible.
• Disallow posting comments after, say, 30 days.

The latter appears to work at the individual entry level. Three of my entries have become particularly troublesome (for some unknown reason). So I've set them to "no more comments". If you find a comment of yours bounced then please do let me know. I'm sure we can fix it.

Deleting all the SPAM comments that have arrived so far for these three posts has been a right pain in the neck. I'm sure a little Firefox Extension programming could make it easier.

And now back to some realcontent. I hope.

Categories : [   SPAM  ]

Jan 28 2008, 04:48:30 AM EST Permalink

ERBSCAN / ERBSHOW - A Parameter You Might Not Know

• ERBSCAN, issued in ISPF 3.4 against an SMF data set, presents you with a list of SMF records and some VERY high level information. Such as the date and time the record was cut, the SMF ID and the record type and subtype.
• ERBSHOW nnnn, issued from the ERBSCAN record list, formats the record. For "non-RMF" records it basically formats the header and presents the record sections - with little additional information. With "RMF" records (equivalently those from BMC's CMF) it does a fair amount more. This is because it knows the layouts of such records.

(Actually ERBSHOW knows about a few other record types.)

If, instead of typing "ERBSHOW nnnn" you type "ERBSHOW nnnn X" you get a better layout and more formatting...

Some key fields in the Product Section are translated.

SMF Type 70 Subtype 1 (CPU) records have two section types for PR/SM. One per LPAR (whether active or inactive) and one per logical engine. It's a nuisance to work out which logical engine section belongs to which LPAR. The extra "X" parameter resequences things so that the logical engine sections for an LPAR immediately follow the section for the LPAR itself. Neat!

Thanks to Matthias Gubitz of RMF Development for sharing this with me. He mentioned it when I started going "wouldn't it be nice if ERBSCAN and ERBSHOW..."

Actually, I took a look at the ERBSCAN / ERBSHOW code. They are a pair of REXX execs that sit in SYS1.SERBCLS. So I copied them into a CLIST library ahead of SYS1.SERBCLS for my TSO / ISPF userid. Then I started to fiddle with the code. It was VERY easy to understand it to the point where I could straightforwardly format an additional field in the record - the Sysplex name.

Modifying the code is, of course, completely unsupported. So I'm not sure I can recommend it. But I do hope to modify it some more - to make job easier. And if I do I may try and give my changes back to RMF Development.

So, while I'm in the code, are there any enhancements to ERBSCAN / ERBSHOW that you would like to see? I can't promise anything, of course. But I'm motivated to work on useful enhancements.

And, no, I don't use ERBSCAN / ERBSHOW instead of serious performance analysis tools. But as the developer of some of these I often need to dump raw records, particularly if I'm trying to trouble shoot or extend our code to map additional fields. I appreciate not a lot of people are in this position.

Categories : [   ERBSCAN  |  ERBSHOW  |  RMF  |  SMF  ]

Jan 11 2008, 05:38:48 PM EST Permalink

Outstupided - Part Umpteen

Thanks to Debra and family for the bath bomb...

Rather than throw it in the bath wrapped I chose to cut it open and pour the contents in. The result? Rather like bathing with dead flies.

And, no, I was sober at the time.


Jan 02 2008, 05:13:30 AM EST Permalink

An Unexpected Benefit of Pornospam

As I've mentioned before I'm using Roller's "Referer URL" page - for my developerWorks blog - to analyse where my blog hits are coming from. (It's an informal exercise because I can only really work with hits that involve a search engine of some sort - and some 80% of my hits are said to be "direct".)

Recently I've had a rash of "pornospam" comments to my blog. (As previously noted I've a way of hiding these entries from you, dear reader.) This has, in turn, caused search engines to index my blog with various porno terms.

So far so bad. But this does have a positive side effect...

It causes a whole bunch of search engines to show up on my radar that wouldn't have before. For example today's "new search engine" roster is:

• http://search.myway.com
• http://search.aol.com
• http://search1.seznam.cz/google
• http://www.manonazi.com

So when "genuine" searches from these engines take people to my (or other developerWorks bloggers') blogs my analysis code will recognise them.

I suppose there's an analogy in terms of virology, along the lines of "what doesn't kill you makes you stronger".

So, thanks pornospammers! But don't try too hard to help me out.

Categories : [   blogspam  |  pornospam  |  referers  ]

Dec 21 2007, 03:22:27 PM EST Permalink

I've A Little Button To Banish SPAMmers

But I've gotten fed up with having to mark these comments as SPAM. So I added some code to my Firefox extension to create a little button on my "Comment Management" page. Press this button and all the comments at the end of the list of comments for a blog entry get marked as SPAM. And hence disappear from under YOUR nose.

Of course I do get genuine comments. So there's an easy mechanism to mark them as NOT SPAM. And I'll use that. But there are MANY more SPAM comments than genuine ones. It just requires me to scan the list of comments once in a while and mark the genuine ones as not SPAM.

Unfortunately this is part of an IBM Internal Firefox extension - as MOST of its function is of little use to non-IBMers. But I might soon be getting to the point where I want to re-engineer to surface some stuff. In the meantime any other IBM with a developerWorks blog who happens to pass by should feel free to contact me to get a copy of "Commentator" as it's called.

(As to the name "Commentator" it arose from the original function - more than 2 years ago - to spruce up Internal Blog COMMENTing by making it easy to add links and to do various text effects. It's come a long way since then.)

Categories : [   Extension  |  Firefox  |  SPAM  ]

Dec 14 2007, 10:23:28 AM EST Permalink

WPS and MXG

Barry and I discussed WPS and MXG over a year ago. I think WPS has come a long way in this time and is now a very credible alternative to SAS for many applications, having a very high degree of programming language compatibility. As Barry notes, it's available for both z/OS and Windows.

Perhaps I'll even get round to learning SAS one day. I have the blue book already, of course.

Categories : [   mxg  |  sas  |  wps  ]

Dec 12 2007, 04:26:16 PM EST Permalink

Mashinations

As I often say you have to assume that people will want to take your web pages and mash them up in ways you never thought likely. If your web page is hard to navigate, extract material from, etc then people will use other pages and sites as the basis of their mashups. And you will lose traffic / business / kudos or whatever other metric of success you choose to use . Unless it's obscurity you seek.

Here are two simple rules I've become sensitised to as a Firefox extension author...

Use id Attributes On "Structural" Elements

Motivation: To reliably navigate to a particular portion of a page javascript programmers use the getElementById() method. It takes the a string parameter and returns one element whose id attribute matches.

Not having this attribute gets you into "I want the third entryfield on the page. No wait, it's moved to be the fourth" territory. Nasty.

But should you apply this to every element on the page? Not necessarily. But you should think about the structure of the page from the mashup perspective. So a major element like the edit field on your page should probably have an id attribute. (This is a real-world example for me as my Firefox extension parses and injects stuff in a suitable entryfield - for a wide variety of pages. All the "not plain text" in this post was injected programmatically by my extension.)

Use name Attributes on Forms And Form Elements

Motivation: In javascript the document element has an array property "forms". So you can refer to an individual form as e.g. myDoc.forms["postForm"] or even simply myDoc.postForm rather than having to hunt through the forms array for a somehow matching form. But only if you give the form a name attribute. What's more you can refer to the elements in the form directly (e.g. myDoc.postForm.textField) but again only if the element in question has a name attribute.

(In fact the value of using the name attribute extends (in part) to images (<img> tags) and java applets (<applet> tags) but the value is rather less. (Navigating to an image and, especially, to an applet is relatively rare.)

One minor Firefox disappointment: I use the Built-in DOM inspector a lot and it doesn't either show the "formname" property or enumerate the forms property by form name. I wonder why it doesn't when there are perfectly good javascript ways of doing it.

These are simple examples of how to make life easy (or difficult). I'm sure there are many more. The bottom line, though, is to make web pages easy to mash up with other sources of data.

A Note On Javascript And Other Languages

I keep mentioning Javascript, don't I? That's simply because it's the language in this space I'm most familiar with. (It's typically seen in Firefox extensions and in AJAX, particularly in frameworks such as the dojo toolkit (which I've just installed.)) An increasing number of web servers are going to be doing their own aggregation or mashing up. I suspect they'll rely on other languages such as PHP.

But it doesn't matter... The guidelines above apply in just the same way to these server-side mashup languages. (I've just installed PHP on Apache and intend to play around with such mashing up at some point.)

Categories : [   DOMInspector  |  ajax  |  dojo  |  firefox  |  form  |  javascript  |  mashups  |  web20  ]

Dec 10 2007, 03:19:26 AM EST Permalink

Where My Searches Come From

This is really just a test of some Firefox Extension code of mine... but it's kind of interesting in its own right...

The following is a table containing the country-level summary of Search URLs that somehow get a reader to my blog on any given day. (I have another table ranking search terms.)

Most days the USA and India are at the top of the list. And there's usually a smattering of other countries represented, mainly in Europe.

What's interesting is that readers from India seem to be mainly interested in JCL, utilities and DFSORT usage. Whereas those from the USA, Europe, China and Japan appear to be mainly interested in Performance- and Architecture-related topics.

I can't prove this to you but the picture outlined above stays pretty much static.

And then there're the searches explicitly on names of women.

A Great Song For A Great Cause

I saw Queen + Paul Rodgers play this in 2005 in Hyde Park. Actually Paul did nothing - Roger Taylor sang it.

But now they've worked on it as part of their current studio sessions and have rushed it out - for free.

So download, play it, donate and tell your friends....

Get it from here and read all about it.

Now I've Downloaded And Listened To It

I downloaded it and listened to it a few times. As with all Queen-related things it grew on me after a few listenings. I would say it's slower than the original 2005 version, which I wish it wasn't. But on the other hand there's some excellent guitar work from Brian May. No surprise there.

All three of them - Paul Rodgers, Brian May and Roger Taylor - shared the vocals. There's some dispute in our household because I prefer Brian's and especially Roger's voices to Paul's. The rest of the family don't think much of their voices.

Roll on the album.

(And here's the song at its original speed.)

Categories : [   46664  |  aids  |  brian  |  hiv  |  mandela  |  may  |  nelson  |  paul  |  qpr  |  queen  |  rodgers  |  roger  |  taylor  ]

Nov 30 2007, 08:42:01 AM EST Permalink

Memories of DFSORT OUTFIL

In September 1997 DFSORT Release 13 was shipped (to coincide with the release of OS/390 Release 4). It took a nice idea from Syncsort and extended it.

In case you didn't know OUTFIL allows you to read an input data set (and perhaps sort it) and write to multiple output files from the resulting records - perhaps selecting subsets of the records and reformatting them (and differently to each output file). All in a single pass over the input data.

Perhaps people really still don't know about OUTFIL as, while I get many searches that hit my blog for DFSORT topics, OUTFIL is rarely one of the search terms.

There are three features that were then unique to DFSORT:

• SPLIT

  This is a "card dealer" function. The most obvious (to me) use was in BatchPipes/MVS "Pipe Balancing". This is where multiple readers absorb the output of a single writer. (Or the other way around.) In this case DFSORT would write to multiple pipes.

• FNAMES

  This allows you to use any output DD name you like. Consider the following...

  OUTFIL FILES=(01,02),...

  and

  OUTFIL FILES=(GERMANY,FRANCE),...

  The latter is much preferable to the former. (The former generates DD names of "SORTOF01" and "SORTOF02" while the latter uses DD names of "GERMANY" and "FRANCE".)

• SAVE

  (This one is, If I remember correctly, my one contribution to the release. I like to make "helpful suggestions" to the Development team and Frank (Yaeger) kindly puts some of my ideas into the product.)

  Consider the case where you've written multiple subsets of the input records to different output destinations. Suppose you want to write the records that haven't been written to any destination to one. It can be very complicated (and error-prone) to figure out the right INCLUDE/OMIT parameter string to make this happen. SAVE automatically does that for you:

  OUTFIL FNAMES=SPECIALS,INCLUDE=(SMFID,EQ,C'PRD1',AND,...)
  OUTFIL FNAMES=THEREST,SAVE

  So I really like this one.

So those are the bare-bones additions to OUTFIL for Release 13. (DFSORT has since added a lot more function to OUTFIL and I would have to assume Syncsort had done the same.)

I had the chance to run a residency that Summer in Poughkeepsie - and it was a lot of fun. A small team of highly skilled people I'm pleased to call friends and I took a version of our batch-driven SMF analysis code and made a mini batch window out of it. (We do have our own mini scheduler and a batch window topology to play with.) So we got to play with DFSORT Release 13 and the (also new) DFSMS System-Managed Buffering and WLM-Managed JES2 initiators and BatchPipes/MVS and had a fine old time of it - running and measuring and tweaking and doing it all again. We got to do a few presentations based on the results of our playing but never got to turn the foils into a Redbook. (I think we had too much fun "playing sysprog" and the like.)

One feature that proved really useful was DFSORT's ability to detect when BSAM or QSAM was needed instead of EXCP. The most common cases are BatchPipes/MVS pipes, Extended Format Sequential (Striped and/or Compressed) and SYSOUT data sets.

Note: I didn't say "QSAM". So that ruled Hiperbatch out. To use Hiperbatch you have to write your own E15, E32 or E35 exit to close the data set and to reopen it for QSAM. (There was a sample piece of Assembler code to do this in the HBAID manual.)

From my (largely Performance) perspective, though, OUTFIL is all about avoiding repeated reads of the input data set. Our batch window did a fair amount of that because it read SMF data repeatedly. So OUTFIL fitted nicely into our window. (And if we pretended the output data was VB and not VBS we could get away with piping it as well.)

One thing to be clear about - which I soon realised - was that OUTFIL does not replace multiple sorts with a single one - unless the sort keys etc are identical. But you could feed the same records from a DFSORT OUTFIL job through multiple pipes into the appropriate number of DFSORT SORT jobs. n sorts become n+1 jobs. More balls to keep in the air, of course. And if those sorts are big enough they'll compete for memory. Which is where another Release 13 feature came in handy: Dynamic Hipersorting. This changed DFSORT Hipersorting from asking MVS (via STGTEST SYSEVENT) how much storage it could have once at the beginning of the sort for sort work to asking the question several times over (as the data was read in). Because of this change Dynamic Hipersorting was much less likely to cause overcommitment of memory - in the multiple concurrent sorts case.

So, to me, DFSORT OUTFIL is yet another of those techniques that you have to "engineer in". Unless you plan the implementation with the usual diligence nothing will happen. We had a lot of fun finding cases where it would be ideal in customer workloads - when conducting PMIO Batch Window studies. And, like so many other techniques, it's as valid today as it was 10 years ago.

One final thing: I've just remembered that I had another idea that got put into DFSORT Release 13...

The PMIO team worked with DFSORT Development to enhance the SMF 16 record (one per DFSORT invocation) to add a few minor fields - in Release 12. (I even forget what they were.) In Release 13 the record was enhanced to contain input and output data set sections - one per data set. These were very detailed - including the number of records read or written, the access method (including Pipes) and so on. Very nice. To get this additional data you need to run with SMF=FULL. (I'd recommend this anyway.)

Categories : [   BSAM  |  BatchPipes/MVS  |  Compression  |  DFSMS  |  DFSORT  |  EXCP  |  HBAID  |  Hiperbatch  |  Hipersorting  |  OUTFIL  |  PMIO  |  QSAM  |  SMF  |  SMF16  |  SMF=FULL  |  STGTEST  |  SYSEVENT  |  Sequential  |  Stripes  ]

Nov 25 2007, 05:26:36 PM EST Permalink

Memories of Pipes

By couching these posts in terms of "memories of" it sounds like they're perhaps obsolete. With the possible exception of Hiperbatch that probably isn't true. (And the only thing really wrong with Hiperbatch is its non-support of Extended Format VSAM and Sequential data sets.)

So, back to the topic - BatchPipes/MVS, usually shortened to Pipes...

The concept of record-level interlock really wasn't new, even at the time... Unix had had pipes for at least 15 years before that, probably 20. In 1990, however, there was a good reason to introduce it to MVS/ESA (as it then was called)... A big customer wanted it. Pipes was born as a result of an exec-level challenge from a specific customer in the USA.

The idea is quite simple... Pipe individual records from a writer job to a reader job - with minimal changes to the application. (In this case "minimal changes" meant changing the DD card to point to your Pipes subsystem.) But I've already written about this.

So, when I was in Poughkeepsie for the second burst of writing the "Storage Configuration Guidelines" Redbooks in November 1990 my new-found friend Ted Blank (who was to become a very good family friend a little later on) told me about Pipes. It was more or less the same conversation that led to the "Parallel Sysplex Batch Performance" Redbook (SG24-2557) because it ranged widely onto more general aspects of batch performance.

Pipes was readied for market through until 1993 or so. At that time in IBM a "new entrepreneurial spirit" was abroad in IBM. Rather than make Pipes a MVS/ESA component it was decided to market it as a specific product. (Personally I think this was a mistake - in terms of aceptance and ultimately customers' batch windows). So the idea was to offer a service to analyse customer data and then lead them into implementing this chargeable product. The data analysis though was more or less restricted to finding "one writer one reader" patterns in the SMF data. True "engineering in" (which is what is called for) happened after this initial screening - if it happened at all.

In parallel (if you'll pardon the pun) we were starting to build the PMIO Batch Window offering. Our whole premise was to engineer whatever it took into workloads, acknowledging the value of rescheduling to run in parallel, breaking up jobs, and "pacing". All things you'd need to really get value out of Pipes. So we, in the PMIO team, were fellow travellers seeking to make Pipes successful and build a good Batch Window Tuning practice. (In the middle of this my manager was offered the opportunity to act as an agent for Pipes in Europe. He declined that offer. He's no longer in IBM, either. Life could've been different.)

I had a great time in the 1990's "chalking and talking" on Pipes. It taught me I could take a complex topic like Pipes and keep it in my head and chalk and talk it. Who needs foils? And sorry if you were a victim of my extemporisation on Pipes in that era.

What was also nice was when DFSORT automated the detection of when EXCP wasn't appropriate for sequential I/O...

Not only Pipes but also Extended Format sequential. This means striped and compressed data sets.

In the Summer of 1997 DFSORT Release 13 came out with this support. (Perhaps I should do another piece called "Memories of DFSORT".)

At the same time - Summer of 1997 - the Pipes team teamed up with BMC, incorporating the latter's Data Accelerator and Batch Accelerator into SmartBatch. Also CMS Pipelines became available (in the main) as a set of "fittings" called BatchPipeWorks. (You specify them on the file DD as a parameter string to the subsystem. Perhaps I should blog on this as well.) And also BatchPipePlex - which routes pipes through the coupling facility. Neither BatchPipes/MVS nor Smartbatch sold well. But I still think the Pipes approach remains valid and valuable.

Now today we have BatchPipes/MVS Version 2 Release 1 - which comprises the original Pipes function, BatchPipeWorks and BatchPipePlex. (BTW folks that's the only way to use "comprise" in sentence.) I also see cases where products consider prereq'ing Pipes - or at least offer support as an option.

And I believe I can STILL extemporise on Pipes. So if you need me to (and preferably if you're on my patch) please get in touch.

So it sounds like I've given myself two more topics to blog on:

• Memories of DFSORT R.13
• BatchPipeWorks

The latter would require me to fire up Pipes on some system or other again. I'm looking forward to playing.

Categories : [   batchlsr  |  batchpipes  |  batchpipes/mvs  |  bsam  |  dfsort  |  hiperbatch  |  pipes  |  qsam  |  vio  ]

Nov 25 2007, 05:16:39 AM EST Permalink