90
with Tags:
batch
X
X
All posts
Batch Architecture, Part Zero
I'm not an architect. I don't even play one on TV. In fact real architects would probably say I'm in the babble phase, architecturewise . But I've been involved in a few situations over the past year or so (and I'm involved in a couple starting round about now) which have led me to the following simple conclusion: Many installations would benefit from drawing up a Batch Architecture . I don't... [More]
|
You Might Just Be A Clone If...
As previously discussed I'm often in a situation of trying to make sense of a set of job-related SMF data. Even though it may be your own installation's data, you're probably confronted with what I like to call “a journey of discovery” occasionally, too. I'm always looking for what I can discern from the data. 1 And, when confronted with a set of data about batch jobs, I go into overdrive. This... [More]
|
Finding The DB2 Accounting Trace Records For an IMS Batch Job Step
When tuning DB2 batch it's important to know which SMF 101 Accounting Trace record corresponds to which job step. A few years ago I wrote code to do this. It works fine for all z/OS DB2 Batch except that originated by IMS. Here's how it works: Find all the Type 30 Step-End records for a given job name. Find the Type 101 Accounting Trace records for which the Correlation ID in the 101 record... [More]
|
Batch Architecture, Part Three
Up until now I haven't talked much about DB2, except perhaps to
note it's a little different. But what is a DB2 Batch job anyway? It's
important to note a DB2 job ISN'T necessarily exclusively DB2 - although
some are. It's just a job that has some DB2 in it. The reason for writing a separate post, apart from breaking things
up a little, is because batch jobs with DB2 in them present... [More]
|
Batch Architecture, Part Two
I concluded Batch Architecture - Part One with a brief mention of inter-relationships and data. I'd like to expand on that on this part. Often the inter-relationships between applications are data driven - which is why I'm linking the two in this post (and in my thinking). But let's think about the inter-relationships that matter. There are four levels: Between applications. Between jobs. Between... [More]
|
Batch Architecture, Part One
First a word of thanks to Ferdy for his insightful comment on Batch Architecture, Part Zero . And also to my IBM colleague Torsten Michelmann for his offline note on the subject. As I indicated in Part Zero I hoped to talk about jobs in a subsequent post. And this is that post. In particular I want to discuss Viewing jobs as part of distinct applications, and Generating a high-level understanding... [More]
|
We Have Residents!
Back in May I wrote about a new batch residency planned for this October and invited good people to apply to join the team. It's been very pleasing how many people applied to be residents and the high quality of the entrants: It was genuinely difficult to pick the eventual team. We also had to reduce the scope a bit - which was a disappointment to both Frank Kyne and I. So, if you didn't get... [More]
|
Data Collection Requirements
Over the years I’ve written emails with data collection requirements dozens of times, with varying degrees of clarity. It would be better, wouldn’t it, to write it once. I don’t think I can get out of the business of writing such emails entirely but here’s a goodly chunk of it. Another thing that struck me is that the value of some types of data has increased enormously... [More]
|
New Batch Residency
In October Frank Kyne and I expect to run a residency in Poughkeepsie.
You can find the announcement here . The residency builds on the ideas presented here and three subsequent posts. I revisited a specific part of it in
Cloning Fan-In . So what are we going to do? For a start we're going to assemble a team of 4 skilled mainframe folks from wherever we can. One of them will be me, which leaves... [More]
|
Evernote, Remember The Milk, SMTP / MIME and z/OS Batch
Another kernel popped the other day: SMTP / MIME . But what on earth is MiGueL Mainframe troubling himself with SMTP / MIME for?
Let's come at this from a different angle... You probably know by now that when you send me your data it gets put through some batch reporting:
Ultimately I don't create the graphs by hand, but I do do the analysis and put the presentation together myself.
That's the... [More]
|
Hackday X - Batch Analytics Baby Steps
Hackday X was good clean fun yesterday - though I think it deserves more than one kiss. Seriously, for once I think I have a hack that actually worked - at least up to a point. I called my entry "z/OS Batch Analytics Baby Steps" and I think that's about right.
My purpose in taking part in successive Hackdays has been more to participate rather than to win any prizes, and to wave the flag a little... [More]
|
Games You Can Play With Timestamps
I've written extensively in the past about what you can glean about batch suites from SMF, most notably SMF Type 30. While I don't believe SMF alone can give you the full dependency network (complete with validation) I've just added some analysis to my code that gets me a little closer. As you're probably never going to run my code the bit that would be interesting is the kind of inferences it's... [More]
|
Two Kinds Of Pipes
It's been a while since I last posted here. And that's been more a matter of being incredibly busy than anything. Some of you will have seen me "on stage" - there having been several conferences and user groups recently. Some of you were with me at the zChampions meeting and some of you I'm actively working on customer situations with. (By the way one of the rules of zChampions isn't... [More]
|
I Said "Parallelise Not "Paralyse" Part 4 - Implementation
Now with free map , this is the concluding part of a four part series on batch parallelisation, with especial focus on cloning. In previous parts I discussed: Motivation Classification Issues This part wraps up with thoughts on implementation. I'm going to break it down into: Analysis Making Changes Monitoring While there probably are iterations of this, this is the essential 1-2-3 sequence within... [More]
|
I Said "Parallelise" Not "Paralyse" Part 3 - Issues
Part 1 and Part 2 were, in my opinion a little abstract. But I think they needed to be: They set the scene for why parallelising your batch can be important. They gave some vocabulary and semantics to help structure our thoughts. Now we need to go a little deeper. Let's start with how to think about the problem of making a job or set of jobs more parallel. Hetereogeneous Parallelism Here the trick... [More]
|
I Said "Parallelise" Not "Paralyse" Part 2 - Classification
I hope you don't get the idea I'm overly into rigour, talking about Classification. But I think it has to be done - to provide terminology for this series of posts. This is the second of four posts on Batch Parallelism, following on from
Motivation . If I think about how parallelism works in batch it broadly falls into two camps: Heterogeneous Homogeneous ( If you look these two terms up in... [More]
|
I Said "Parallelise" Not "Paralyse" Part 1 - Motivation
I have enormous trouble pronouncing "parallelise" right - and not saying "paralyse". It's true, and I bet many of you have the same trouble (sober or not). It's on a par with "red lorry yellow lorry" or "the Leith Police dismisseth us". But it's a word I think we're going to have to get used to pronouncing right. And this post will explain why. This looks to... [More]
|
CICS and Batch
In my experience there are two kinds of CICS installations: Those that take CICS down at night - to run the Batch - and those that don't. There is a loose correlation between what the data manager is and which approach is taken: VSAM-based CICS applications tend to be less 24x7 than DB2 ones, though it's not that clear cut. This post is about how you (really I) might glean how you run CICS... [More]
|
DB2 Package-Level Statistics and Batch Tuning
I don't know how many years it's been since DB2 Version 8 was shipped but I've FINALLY added support for some really useful statistics that became available with that release. As so often happens I was caused to open up my code because of some customer data that exposed a problem in it: The customer sent DB2 Version 8 SMF 101 Accounting Trace data that contained Unicode. In particular DB2... [More]
|
Touring The Upcoming Batch Optimization On z/OS Redbook
As mentioned here , I recently participated in a residency in Poughkeepsie. Our task was to write the "Batch Optimization on z/OS" redbook that we hope will get out soon (though I think early next year is the most realistic timescale). These past two weeks I "toured" the redbook in the Nordics: 5 cities in 2 weeks (Helsinki, Copenhagen, Oslo, Stockholm and Aarhus). I really... [More]
|
