Most organizations agree that encryption must be part of their overall strategy for protecting and securing sensitive data. They also recognize that protecting only the data that is required to achieve compliance is a minimum threshold and that a move from selective encryption (protecting only specific types of data) to pervasive encryption (encrypting all data) is needed. Likewise, many barriers that are encountered today with current enterprise data protection policy and strategy can be removed with pervasive encryption, such as:
Decoupling encryption from data classification; this allows organizations to implement their encryption strategy independent of any challenges they might face while identifying and classifying sensitive data. It also reduces the risk of unidentified or mis-classified data.
Using encryption without interrupting business applications or affecting service level agreements (SLAs); changes to the application are not required if data is encrypted after it leaves the application and decrypted before it reaches the application.
Reducing high costs that are associated with processor overhead; the cost of encryption is minimized by encrypting data in bulk, by using encryption accelerators with high performance and low latency.
Pervasive encryption on IBM Z platforms covers extensive encryption of data at-rest and data in-flight. Data at-rest is data that is written to and stored on devices, such as disk and tape; data in-flight is data that is sent over the network to a user or another platform, or over a storage area network to disk and tape devices.
IBM Z pervasive encryption is attained through tight platform integration of hardware, firmware, and software. It simplifies the implementation of data encryption and reduces the cost that is associated with protecting data and achieving compliance. Specifically, z/OS data set encryption provides broad coverage for sensitive data residing in supported data set types by using encryption that is tied to access control for both in-flight and at-rest data protection.
z/OS data set encryption is enabled through policies and profiles. Data can be encrypted in bulk with low overhead, while allowing for varying levels of granularity. Operating system components and Z hardware integrated cryptographic engines deliver industry exclusive protected key encryption with high performance and high security, including:
z/OS Integrated Cryptographic Services Facility (ICSF) to manage encryption keys and the key labels that are associated with the encryption keys that are in the Cryptographic Key Data Set (CKDS).
SAF or RACF profiles and SMS policies to control allocating encrypted data sets by associating key labels to those data sets. In addition, SAF or RACF profiles can control access to individual data sets or groups of data sets, and the key labels.
The Central Processor Assist for Cryptographic Function (CPACF) provides high-speed cryptographic acceleration through a set of instructions that are available in hardware on every processor unit.
The Crypto Express adapter provides cryptographic function through high-security, tamper-responding hardware security modules (HSMs) to safeguard encrypted data by protecting encryption keys.
z/OS data set encryption is also transparent to applications and allows for the separation of duties within an organization. Because data remains encrypted (even during operational procedures), z/OS data set encryption can remove the need to include storage administration as part of the compliance scope. The use of more compliance controls might not be needed because the data remains encrypted when it is written.
The data set types that are supported by z/OS data set encryption are extended-format sequential data sets and extended format VSAM data sets, which can then be used by z/OS zFS, IBM Db2, IBM IMS, middleware, logs, batch, and Independent Software Vendor (ISV) solutions. Applications or middleware that use extended-format data that is accessed with VSAM, QSAM, or BSAM access methods also can take advantage of z/OS data set encryption.
The figure below shows how z/OS data set encryption works, including the flow between the hardware, firmware, and software components. This example involves input processing of an encrypted data set. It uses secure encryption data keys that are protected by Crypto Express and stored in the CKDS. The data key that is used to encrypt and decrypt the data is wrapped as a secure key or protected key during the process. Note that the data key material is never visible to the operating system or application.
The left portion of the figure shows the following steps that occur during the data set open process:
1. DFSMS receives the key label that is associated with data set from the catalog and calls RACF to verify the user’s access to the key label.
2. DFSMS calls ICSF with the key label.
3. ICSF obtains the secure key from CKDS and calls the Crypto Express6S to unwrap the key.
4. With assistance from the firmware, Crypto Express6S decrypts the secure key and rewraps with a transport key.
5. The wrapped key is sent to CPACF. With assistance from Z firmware, CPACF unwraps the wrapped key with the transport key to make available the data key.
6. The data key is wrapped with the CPACF wrapping key to create the protected key.
7. The protected key is sent to ICSF, where it is cached in protected memory for future callers. ICSF sends the protected key to DFSMS to encrypt and decrypt data.
The right portion of figure shows the steps that occur during the data set read/get process:
A. DFSMS reads the encrypted data from data set and initiates CPACF. The protected key is then passed.
B. CPACF decrypts data by using the protected key.
C. Decrypted data is sent as clear text to the application through DFSMS.
Today’s security requires consistent protection against threats and malware. Enterprises must be flexible while having a secure infrastructure to protect effectively the most valued asset of a company (the data), and their access through the cloud. Running many distributed servers involves much effort to install, manage, maintain, and provide security for them. To contain this effort, many enterprises are consolidating these servers on z Systems or LinuxONE by using the z/VM as the hypervisor, taking advantage of the virtualization technologies to use the hardware effectively and to simplify administration tasks.
It is generally held that “security through obscurity” is not a valid method. Using open, well-established security methods implemented correctly provides the best defense. For example, instead of developing your own cryptographic libraries, you should instead use open, established ones that have been vetted for many years. Hiding information creates more system administration work and any mistakes may fail to protect against attacks.
Implementing the enterprise security policy and following the least privilege principle increases the strength of security in your enterprise cloud.
In a LinuxONE environment, the building blocks of the Cloud environment could include:
The z/VM Directory Manager (DirMaint),
Extreme Cloud Administration Toolkit (xCAT),
z/VM Cloud Manager Appliance
CMA allows the usage of OpenStack to deploy Linux guests on z/VM, and for the integration of z/VM into larger environments. The CMA version is upgraded to OpenStack Liberty and is fully supported as a z/VM component without additional license requirements. CMA only manages z/VM platforms and it does not deploy guests onto non-z/VM platforms. The CMA changes provide several different options for using CMA, either as stand-alone cloud or integrated with another OpenStack environment.
Lydia Parziale is a Project Leader for the ITSO team in Poughkeepsie, New York, with domestic and international experience in technology management including software
development, project leadership, and strategic planning. Her areas of expertise include business development and database management technologies. Lydia is a certified PMP and an IBM Certified IT Specialist with an MBA in Technology Management and has been employed by IBM for 25+ years in various technology areas.
Before you begin creating and provisioning virtual machines for your business applications, ensure your KVM hypervisor is capable of sustaining these applications. All the virtual machines will rely on the hypervisor's integrity and availability. The three key areas to initially address are:
Protecting data and resources by providing secure tracking, audit trails, alert mechanisms, and reporting
Managing and monitoring resources, and offer interfaces for configuring or modifying virtual machines as needed
Backing up data and executing data recovery aligned to the application's needs.
Protecting data and resources
KVM hypervisor security is critical because it typically has access to all of the virtual machines' resources under its control. If the hypervisor is compromised, an unauthorized user could potentially gain access to confidential data. Good security practices are essential for establishing business trust. How do you do this? Several open source and commercial tools can help effect good security practices and policies. To establish consistency, the same tools available to secure a KVM environment can also secure Linux virtual machines.
Some of the key software components to secure the KVM hypervisor are:
FirewallD for network security
LDAP for centralized authentication
SELinux for access control policies that confine access to data
Linux Audit to provide detailed audit trail information you might not find in the system log.
In addition, there is support for cryptographic hardware in the IBM z Systems platform that can perform DES, TDES, AES, RSA, SHA-1, and SHA-2 cryptographic operations. CP assist for cryptographic functions (CPACF) instructions are available to KVM for IBM z and its Linux virtual machines when the kernel modules are loaded.
Managing and monitoring resources
The Linux ecosystem offers open source and commercial monitoring tools by which the KVM for IBM z resources can be managed and monitored. There are three primary methods that can be used:
The Linux shell in KVM for IBM z is available to handle most any resource configuration. CPUs can be configured on or off, memory can be enabled or disabled and storage devices and network interfaces can be added or removed.
Utilizing the IBM z Systems HMC, either with DPM mode or with standard PR/SM mode additional processors or memory can be dynamically added to a logical partition. With DPM mode, additional storage devices and network interfaces can be added or configured dynamically.
Kimchi’s management interface for KVM is http based. It allows for KVM network and storage resource management.
KVM for IBM z provides a number of built in open source monitoring packages such as nagios monitoring plugs, snmp agents, standard libvirt APIs, sar, systemtap, and many more. And if you find what was provided does not exactly fit your needs, KVM for IBM z Systems does provide an SDK. The SDK has the compilers and development libraries need to build perform builds of additional software projects.
Libvirt is a library of open source APIs that includes a daemon and management tools, that are installed with KVM for IBM z. You can create, delete, run, stop, and manage your virtual servers using the virsh command. Besides virsh, there is a graphical tool called Virtual Machine Manager or more commonly “virt-manager”. Virt-manager can handle most of the common lifecycle functions of a virtual server, including installation. It also has basic monitoring, console access, and resource management of the virtual server and some KVM host resources.
Many open source tools are typically included in Linux distributions, and if they are not included you can build them from source. To maintain a consistent approach, chose tools that manage both the KVM hypervisor and it's virtual machines.
Backing up data and executing data recovery
A KVM for IBM z environment can be backed up in a number of ways, therefore when designing your backup and recovery strategy consider the following questions:
Should the virtual machines to be up and running or require them to be shutdown during the backup and recovery?
How is the disk storage provisioned to the virtual machine?
What is the recovery point objective (RPO)?
What is the recovery time objective (RTO) ?
The KVM hypervisor and virtual machine backups can be categorized as:
The core operating system disk needed for boot
The additional storage used to host image files and system logs
Key configuration files such as for networking and virtual machine definitions
There are multiple ways to back up each of these categories. The core operating system disk could in its most basic form be backed up via Linux dd commands from another system. You might want to do this right after installation. You could also utilize FlashCopy or disk mirroring technologies to create a consistent point in time copy without taking down the KVM hypervisor or virtual machine. To exploit FlashCopy or similar technology, there typically is a requirement to install some command line interface program to direct the FlashCopy operation and to have network connectivity to the console of the storage subsystem.
The additional storage used to host image files could also use FlashCopy or disk mirroring, but other options exist as well. A QCOW2 snapshot or a LVM snapshot are examples of other options that may help you minimize downtime.
Key configuration files such as the KVM hypervisor network definitions, Open vSwitch definitions, zipl.conf, zfcp.conf and others could be backed up via file based tools such as rsync. The amount of storage these files take is relatively small.
It may also be useful to have partition, volume group, LVM, and file system information captured and recorded in the event you need to perform a recovery. This information could be easily gathered on a regular basis and transmitted to a remote archive.
Another option would be to utilize file level backups either with open source tools like rsync or commercial tools like IBM Tivoli® Storage Manager (TSM). If a virtual machine were destroyed one approach might be to provision a new base Linux and restore all the files from the most recent backup, rather than using disk image level backups and restores.
Part of the planning for backup and recovery also needs to consider the middleware. For example a database would typically utilize its own utilities in order to provide backups without any or minimal down time. A comprehensive backup and recovery strategy typically involves multiple backup methods and the recovery from those backups should be regularly tested.
To help make you plan and deploy a successful and effective environment, read Getting Started with KVM for IBM z Systems, SG24-8332 at:
Bill White is an IBM Redbooks Project Leader for IBM z Systems. He works with technical experts from around the globe to produce technical enablement content.
This week's guest blogger is Ravi Kumar. Ravi is a Senior Managing Consultant at IBM (Analytics Platform, North American Lab Services). Ravi is a Distinguished IT Specialist (Open Group certified) with more than 23 years of I/T experience. He has a Masters degree in Business Administration (MBA) from University of Nebraska, Lincoln. He had contributed to 7 other redbooks in the areas of Database, Analytics Accelerator and Information Management tools. His social profile can be
viewed at: http://www.linkedin.com/in/ravikalyanasundaram
IBM SPSS Modeler is a powerful analytic tool that supports all phases of data analytics process, including data preparation, model building, deployment, and model maintenance. You can leverage SPSS Modeler to build analytical models, which can be used in statistical analysis, data mining and machine learning. The data scientists can work with user-friendly SPSS Modeler client interface to access mainframe data with the same level of ease as that of data from any other platform they are accustomed to. SPSS Modeler can also take advantage of in-database transformation and in-database modeling using IBM DB2 Analytics Accelerator for z/OS (IDAA) as the data analytics hub on z/OS.
Until recently, z Systems did not offer an efficient solution in the area of complex mathematical processing. So, in the past, you may have resorted to the idea of offloading operational data (that is a snapshot from a prior point in time) from z Systems to a distributed platform in order to implement machine learning, and those solutions often resulted in obsolete and unreliable results in addition to the unwanted security exposures.
Now, with IBM DB2 Analytics Accelerator you can enable Machine Learning on your OLTP applications that produce and consume z Systems data, simultaneously accelerating the execution of data transformation and analytical modeling processes with the power and performance of MPP (Massively Parallel Processing) architecture in IBM Netezza appliance. All without offloading data from z Systems to distributed environments (which by the way, also eliminates a potential data breach situation).
In-transactional scoring using the Predictive models created with the above approach can scale with your DB2 for z/OS transactional environment. This is accomplished through in-database scoring using SPSS Scoring Adapter for DB2 for z/OS, which perform real-time scoring on your predictive models to quickly reveal what's interesting in your data. When the predictive model is published in SPSS, the Scoring Adapter for DB2 z/OS uses PACK/UNPACK functions for efficient parameter move and can create an SQL statement with HUMSPSS.SCORE_COMPONENT UDF. This generated SQL statement can be embedded in your OLTP application. The other popular alternative is to generate scoring model in open-standard PMML (Predictive Model Markup Language) format. The score can then be combined with your business rules to make real-time decisions on your DB2 for z/OS data from within your mainframe applications. You may also resort to vendor tool called Zementis that uses the generated PMML to implement in-application scoring in CICS and Java applications accessing DB2 for z/OS.
The above approach easily enables your OLTP and batch applications accessing mainframe data with early machine learning capability to learn hidden patterns in your operational data using mathematical modeling algorithms that are readily available with IDAA (as INZA stored procedures that entirely runs on the Accelerator). With IDAA V5.1, you can utilize five major predictive analytics algorithms viz., K-Means, Naive Bayes, Decision Tree, Regression Tree, and Two-step.
Unsupervised Learning algorithms like K-Means and Two-step uses descriptive statistics to analyze the natural patterns and relationships that occur within your operational data on DB2 for z/OS. Unsupervised learning models can identify clusters of similar records and/or relationships between different fields within an accelerated DB2 for z/OS table. For example, K-Means and Two-Step clustering algorithms (available through stored procedures like INZA.KMEANS and INZA.TWOSTEP) can enable Machine Learning in areas like market segmentation, geostatistics, market basket analysis (by association learning) and so on.
Supervised Learning uses historic/training data to construct decision trees and the constructed tree is then used to predict future values. Classification technique can be used to identify which group or type a new record, that is being inserted into your DB2 for z/OS table, belongs to based on key characteristic values on its fields. Regression technique can be used to predict future values for a given field based on past historic values. Algorithms like Naive Bayes, Decision Tree, and Regression Tree can be used to solve classification and regression problems. Thus the predictive models using supervised learning algorithms (available through stored procedures like INZA.DECTREE, INZA.REGTREE, and INZA.NAIVEBAYES) can be used to predict whether a customer will buy or leave, credit card fraud, up-selling opportunities, voters responsiveness to different types of election campaigns and so on.
Summary: Neuroscientists say that pattern recognition and emotional tagging help humans with quick decision making. Algorithms are a big part of machine learning and these algorithms can aid the executives with more and more evidence based decision making using hot operational data on z/OS. The executives can now combine modern machines' processing power with their own ingenuity to avoid flawed decisions that are sometimes caused by emotional tagging.
Today, we’re delighted to share the latest member of the IBM z Systems family: the IBM z13s. We think you will like it. A lot.
The z13s delivers many exciting possibilities over its predecessor, the IBM zBC12.
The short list includes:
Accelerated data and transaction serving
Integrated analytics for insight
Access to the API economy
An agile application development and operations environment
Efficient, scalable, and secure cloud services
End-to-end security for data and transactions
The high levels of virtualization provide options for cloud deployment to assist with such areas as application development and testing. The hypervisor is key for virtualization and the z13s supports both hardware and software hypervisors (PR/SM, KVM, and z/VM).
The underlying architecture has expanded to enable new solutions such as integrated analytics to bring valuable opportunities for your business while support existing applications.
N10: 1 CPC drawer with max. of 10 customizable PUs
N20: 1 or 2 CPC drawers with max. of 20 customizable PUs
Up to 4 TB
The z13s offers 26 capacity levels (times 6 CPs) for 156 settings. Plus, models are offered for either all-IFL or all-ICF configurations.
Analytical vector processing
Redesigned larger caches
Enhanced accelerators for data compression and cryptography
For enterprises aiming to move their IT infrastructures in closer alignment with their business plan, the z13s offers unparalleled levels of flexibility through virtualization, analytical insight, security. Enterprise-wide agility will help you embrace the challenges of the exploding on-demand digital age.
In the course of an IT career, many of us may have sat at our desks looking at a sluggish application and wondered, "If I increase the amount of memory here or there, will this improve performance?" And, hopefully, your next thoughts would have been about the impact on I/O operations and cost, CPU usage, and transaction response times.
Although the magnitude of these changes can vary widely based on a number of factors, including potential I/Os to be eliminated, resource contention, workload, configuration, and tuning, you should carefully consider whether your environment could benefit from the addition of more memory to your software functions.
Significant performance benefits can be experienced by increasing the amount of memory assigned to various functions in the IBM® z/OS® software stack, operating system, and middleware products. IBM DB2® and IBM MQ buffer pools, dump services, and large page exploitation are just a few of the functions whose ease of use and performance can be improved when more memory is made available to them.
Recently, an IBM Redbooks Redpaper was published that can help you to examine the performance implications of increasing memory in the following areas:
DB2 buffer pools
IBM Cognos® Dynamic Cubes
MDM with larger DB2 buffer pools
Java heaps and Garbage Collection tuning and Java large page use
MQ v8 64-bit buffer pool tuning
Enabling more in-memory use by IBM CICS® without paging
DFSort I/O reduction
Fixed pages and fixed large pages
Different environments, of course, may experience a wide range of performance benefits but there does seem to be enough evidence to suggest that configuring more memory could be a positive enhancement for many installations due to reduced I/O rates, improving transaction response times, and in some cases, reduced CPU time.
To read more about this and see some examples, read the IBM Redbooks Redpaper :
A year and a half ago, IBM Wave for z/VM came onto the scene to provide a simplified and cost effective way for companies to harness the consolidation capabilities of the IBM z Systems platform and its ability to host workloads of tens of thousands of commodity servers. In December 2015, an IBM Redbooks residency started running to make important updates to the IBM Redbooks publication, IBM Wave for z/VM Installation,Implementation, and Exploitation, SG24-8192. IBM Wave Release 2 further expands the capabilities by delivering increased support for Linux distributions and devices, as well as additional enterprise-grade security and performance enhancements.
Some of the updates in this book include instructions on how to do a bare metal installation from Red Hat Enterprise Linux Servers using IBM Wave for z/VM.
Additionally, this IBM Redbooks publication includes a new chapter that describes IBM Wave / BTS parameters that might influence performance and resource usage. This chapter discusses:
The IBM Wave Parameters window
The BTS Manager window
How to restart the Background Task Scheduler (the BTS)
And how to produce a dump of the BTS
We’ve also included an appendix in this version of the IBM Redbooks publication that includes, among other things, IBM Wave for z/VM flow charts that can assist you in planning, preparation, installation, and setup of your IBM Wave for z/VM system.
For to download this IBM Redbooks publication, see:
The biggest reason to split the books is that this will allow us to update books as new versions come along instead of waiting. It will allow for our resident teams to work more in depth on each volume to provide a deeper dive into the content of each volume. Additionally, if you only want to learn more about one of the volumes, you can just download that volume. It's a more streamlined way of getting and finding the content you need, when you need it.
What are your thoughts on going forward with this publication? Should we merge them back together in the next iteration or keep them separate?
And by the way, if you are looking for the previous version of the IBM Redbooks publication, The Virtualization Cookbook for z/VM 6.3, RHEL 6.4, and SLES 11 SP3? You can now find it here:
When running in a virtualized environment, any reasonable administrator tries to reduce the time needed for standard tasks. In the early days of Linux on z/VM, this resulted in a procedure using golden images and cloning. This procedure simplified the deployment of Linux to new z/VM guest systems and has served many administrators well for a long time. However, over time, the Linux systems changed. With the introduction of newer technologies such as systemd on Linux, a number of problems came about that made the once so nifty feature of cloning golden images more and more difficult.
Problem: Make the image golden
During first bootup, Linux creates unique data at lots of locations. The number and location depends on the installed software. It requires detailed knowledge about the software used to make sure, that all these strings are
recreated during the first bootup of the cloned machine.
Unfortunately, there is no means to detect the needed changes available in the system. However having some of those places not updated can result in security issues and data corruption of the involved clones later on. A clone
that works in the first place is not necessarily done right.
This issue is not new, it already existed with SLES11 and RHEL6, however it became worse with the introduction of systemd and its machine id. It is therefore recommended, to move away from deploying clones to use either automated installation or the imaging software kiwi.
Solution: do not create the unique data in the first place
The actual problem exists only, because cloning relies on the configuration of a readily booted system. This system then is cleaned up and prepared for the actual cloning process. After cleanup, it is also called "golden image". All of the files needed within the production system are already created during the first startup of this system. The cleanup process must take care to remove all data from the system that should be uniqe. This data has then to be recreated during the first bootup of the clone.
The only reliable solution to accomplish this is, to avoid the creation of the unique data in the first place. This means, the golden image never should have been booted before cloning new virtual machines. To avoid issues, you may want to use automated installations as described in "The Virtualization Cookbook for z/VM 6.3, RHEL 7.1 and SLES 12". However if you have to rely on readily build images, the creation of virtual appliances is the way to go.
This is where the imaging software KIWI steps in.
Instead of creating a golden image to clone, a virtual appliance is created. This virtual appliance is never booted during the image creation process. The deployment of the virtual appliance is very similar to the one of a golden image: It is copied to a new disk, and given several parameters to finalize its configuration during the first startup.
If your business processes requires you to test a readily built image, this is also possible with the virtual appliance. However, needed changes to the image must be done the the KIWI configuration, and will only be available with the next iteration of a newly created image of the virtual appliance. You don't apply the changes to the live system, but to the configuration of
the virtual appliance.
This procedure can simplify automations. For example, to provide an image with all updates installed, you will just need to provide the update repositories during the image creation. After new updates are available that you need in your golden image, just repeat the building process, and the resulting image will contain all the updates. This also results in more
secure systems at the time of redeployment compared to deploying the updates only after starting the original image.
Our IBM Redbooks blogger, Berthold Gunreben, is a Build Service Engineer at SUSE in Germany. He has 14 years of professional experience in Linux and is responsible for the administration of the mainframe system at SUSE. Besides his expertise with Linux on z Systems, he is also a Mainframe System Specialist certified by the European Mainframe Academy: http://www.mainframe-academy.de. His areas of expertise include High Availability on Linux, Realtime Linux, Automatic Deployments, Storage Administration on the IBM DS8000®, Virtualization Systems with Xen, KVM, and z/VM, as well as documentation. Berthold has written extensively in many of the SUSE manuals.
We would like to introduce you to this exciting new release of the operating system by sharing our IBM Redbooks content with you.
The IBM Redbooks team brought together expertise from around the world to help you discover and explore the potential of z/OS V2R2. Let IBM Redbooks guide you through the opportunities that the new release of the operating system can bring to your business.
We have modularized the content to help you pick and choose subjects which pique your interest. We suggest you start with The IBM z/OS V2R2: IBM Redbooks Content Guide to understand how we have categorized the topics and the related content.
If you already see the topic you want then download it immediately!
I couldn’t help but notice that there was a lot of talking about me lately. And you all know how it goes when people start talking about you. Rumors come up about what you can do and what you can’t. Sometimes people even seem to know you better then you do yourself. So I figured I can’t let this go any further.
But before I go into detail about why you really should be using me, I want to talk some business. I mean, you all see the increasing importance of analytics in business. You need it for your organization’s success, for your customer’s satisfaction and for your own decision-making. But data alone is not enough. You can have all the data you want on your customers, but if it takes you too long to use the data to your advantage you’re at a disadvantage against your competitors. They might leave you behind if you don’t react to the market fast enough. Telling by experience and from all the data I’ve seen, we live in a fast world. Processes have to be adjusted to situations; decisions have to been made quickly.
The last couple of weeks I came across that word “agile” over and over again. Sounds like a great concept to me. But most of all it sounds like people realized that for your business two things are essential: knowledge of what is going on, and using this knowledge to react appropriately. What I can do for you is to form a self-managing hybrid environment together with DB2 for z/OS to address the needs of business intelligence and analytic processing workloads. I even can do that while continuing to run mission-critical transaction processing and analytical workload concurrently and efficiently. I leverage the power of zEnterprise, DB2 for z/OS and Netezza technology which makes it possible for you to integrate analytics insights into operational process to drive business critical analytics resulting in exceptional business value. Come on, you have to admit this is impressive, isn’t it?
Still not convinced of my capabilities? OK, what if I tell you that I can also do all of the following:
Supporting running complex queries on very large volumes of data
Accelerating analytic query response times
Lowering cost of storing, managing and processing historical data
Minimizing the need to have data marts for performance
Reducing capacity requirements on z systems
Reducing operational cost and risk
In combination with DB2 and z/OS I can accelerate data-intensive and complex queries in a DB2 for z/OS highly secure and available environment
And I am quite flexible too! I support the Accelerator-shadow table, the Accelerator-Archive table and the Accelerator-only table. You want to know what that means? Well, explaining all of it right now would go beyond the scope of a blog post and I have a date for lunch. But I heard there would be a Redbooks publication to give you all the information you need… Check it out and we will talk about it later!
Last weekend I spent quite some time thinking about how to create a blog post to get people as excited about DB2 for z/OS temporal data management as I am. After a while I decided to call an old friend to take a break and maybe receive new input. Somehow we ended up playing an old childhood game in which you put each letter of your name in a different line and then find a word for each letter that describes you. In the end you have a list of attributes describing your personality. So why not trying this to describe something technical such as temporal data management as well? Here is what I came up with:
Time-based data management that can help businesses manage the increasing amounts of data and retention requirements
Enables you to accurately track information and data changes over time.
Makes it easy to insert, update, delete and query data in the past, present or future by using new and standardized SQL syntax.
Provides an efficient and cost-effective way to address auditing and compliance requirements.
Opportunity to have multiple stored versions for every logical row.
Remembers all past versions of rows in a table. If we are talking about a bank account for example, DB2 for z/OS temporal data management will help you to provide a detailed history of their accounts to your customers – and not by using additional tables with triggers or stored procedures as is current practice.
Application development, maintenance and management can be simplified.
Leverage DB2 for z/OS temporal data management to obey regulations and fulfill customer needs, no matter if you are in the insurance, financial, retail, human resources or any other sector.
As you can see DB2 for z/OS temporal data management provides many ways to help you and your customers to successfully face today’s business challenges by recording and maintaining ever increasing amounts of data.
So you’ve built a killer application. It’s useful. It’s novel. It’s clever. Surely it’s going to be a huge success – fame and fortune await. Or…. do they?
As IBM Distinguished Engineer Frank De Gilio tells it at the SHARE Orlando conference, usefulness, cleverness and novelty in today’s market are not enough. There are three other important factors to consider:
Is it fast?
Is it efficient?
Is it easy to use?
Never has this applied more than it applies to today’s world, and the role that mainframe applications play in that world. We have many big monolithic mainframe applications and they all live in the data center. The aggregation point is the mainframe servers in that data center. But those days are gone. The days of terminals, and even latops, is disappearing. Mobile is the new aggregation point, and time from development to production has moved from months to days.
This new world is the API economy. Monolithic applications are broken down into smaller pieces – functionality that you can call – known as services. This is the microservices architecture at work – the approach of designing applications as collections of smaller, independent services.
Frank De Gilio describes how this new business programming model can be divided into two roles:
Hardcores – These are the people who understand how the mainframe systems work
It is these Scripters that services of today need to appeal to. Scripters don’t care about platforms, they care about how fast, efficient, and easy to use your service is.
Unleashing current business applications as services provides big advantages, putting existing capabilities into the hands of new users. Combining cloud service model with z/OS parallel system is a winner!
Do you still create utility jobs manually to maintain several objects? Do you think that your maintenance jobs need to be run on a predefined frequency basis? IBM DB2 Automation Tool for z/OS helps you with these challenges.
Combining object, utility, and job profiles, DB2 Automation Tool can reduce and facilitate manual routine tasks and focus on more complex job responsibilities that add more value to your company. Additionally, when using exception profiles and DB2 Automation Tool, you can define in a utility profile when to run a utility against an object in an object profile. You select the conditions from a statistics list in the exception profile.
But instead of talking about the solution itself, we want to give you more information about what these profiles actually are, how they work and how you can use them to create an autonomic infrastructure:
Object profiles allow you to create reusable lists of objects. You can group related objects into one profile, such as all objects for a particular application, objects with similar maintenance requirements, etc. In an object profile, you can include objects on which you want to run utilities, as well as exclude objects that you want the utilities to ignore.
You can create object profiles using either the IBM Management Console for IMS and DB2 or by using the ISPF panels in automation tool. Here you can see the GUI for creating it in the IBM Management Console:
A utility profile is a collection of one or more utilities and their respective run time options. Using a similar technique to creating object profiles, we can now create a utility profile to address any particular maintenance requirement. You can select the utilities that you want to execute and “Update Utility” will allow you to specify the parameters you want to specify for that given utility. Once created a utility profile can be updated at any time to include more utilities or to change the options for a given utility.
The following list shows the utilities and functions that are available:
Exception profiles allow you to define when a utility in a utility profile should be run against an object in an object profile. You select the conditions from a statistics list in the exception profile. The exception profile is placed in the job profile with the object and utility profile. During the job build, exception processing produces a list of accepted objects and a list of rejected objects. When creating utility profiles, you can specify whether the utility is to be executed on the accepted objects, the rejected objects, or both.
There are 184 available selection criteria that we can use to select candidate objects. Also, we can provide our own criteria through a user exit interface. There are 10 supplied default exception profiles and viewing these will give you a good idea on how to create and specify your own based on your site standards:
Job profiles combine the object profiles and utility profiles (and optionally exception profiles) into a set. If no exception profile is included in the job profile, then each utility is run unconditionally on each object on the object list. You can combine multiple object profiles with multiple utility profiles, and can specify the job step order for the generated job. The combined profiles, which are headed by the job profile, form the basis of a DB2 Automation Tool task. You can submit this task manually or schedule it by using the DB2 administration task scheduler or your site’s scheduling software. The job profile will evaluate the exception profile against the objects in the object profile and when a condition is met will generate JCL and Utility statements to perform the tasks specified in the utility profile against the objects that met the condition.
To create a job profile use the ‘C’reate command on the command line:
These profiles allow you to help IT staff reduce think time to repetitive tasks and also to analyze the environment in order to run only what is needed and when it is needed, reducing the CPU utilization for maintenance jobs that do not really need to run in a defined maintenance window. So by combining object, job, exception and utility profiles with the DB Automation Tool, you can make your database environment work more efficiently.
With the addition of the Management Console and the Autonomics Director you can now not only exercise “Passive” autonomics but you can start to move into “Active” autonomics. The Management Console makes monitoring the current symptoms and automating the suggested actions easy.
And how about you – did you already created an autonomic infrastructure? What were your experiences using these profiles in DB2? Tell us what you learned while working with these products. If you want to see additional material about the process of creating the autonomic infrastructure using DB2, see the IBM Redbooks publication Modernize Your DB2 for z/OS Maintenance with Utility Autonomics.
There have been many changes in the past 25 years in our IT world that have led to the need for autonomics in our database environment, especially in DB2 on z/OS. But while always talking about the solutions, the question arises – how do you actually implement them?
Each company may have different priorities which dictate the order of the implementation steps. Company A may need to apply intelligence to their reorg utilities as their top priority while Company B may need to address utility standards because of the impending retirement of the support person for their homegrown DB2 utility generator. Regardless of the order in which you start, IBM provides the software for a comprehensive autonomic environment that addresses the business problems that most companies face: limited expertise, greater application availability, or the need to control costs by moving work to off peak hours.
Here we will show you how to move from the traditional steps into a modernized autonomic environment, by implementing an active strategy for your DB2 Maintenance Tasks, following these five steps:
Step 1: Collect the metrics and related statistics for utility maintenance
First of all, you have to collect all relevant statistical data on your DB2 objects. This data can be used to filter out objects that are physically disorganized. Your goal is to execute run maintenance by exception and filter out wasting of resources attributed to running of utilities. IBM provides two DB2 stored procedures that collects statistics about objects we have defined via a profile and will generate an alert if the statistics exceed criteria, placing the alert into a table, and performs the RUNSTATS for your optimizer needs.
Step 2: Group your objects
Grouping your DB2 objects can be achieved in several ways. DB2 Automation Tool provides a function called Object Profiles which provides more flexibility and functions for object grouping – called Object Profiles. Using these Object Profiles, you can include objects on which you want to run utilities, as well as exclude objects that you want the utilities to ignore. Object Profiles are similar to DB2 TEMPLATEs. They allow table spaces and index spaces to be chosen for processing in much the same way.
Step 3: Create exceptions and thresholds for utilities
The next step to implement an active autonomic strategy is to run all your maintenance by exception filtering. The DB2 Automation Tool provides a function called the Exception Profile. This definition contains the conditions under which users want to run utilities. When combined with Object Profiles and Utility Profiles, the Exception Profiles act as a filter against the objects specified in the Object Profile.
Step 4: Build optimized utility JCL and jobs
Before execution, first you have to build the optimized utility JCLs and jobs. Regarding this, Job Profiles are used to connect the different profiles which are created in the DB2 Automation Tool. A Job Profile is the master profile and associates all the profiles - Utility Profiles, Object Profiles, and Exception Profiles - together. The combined profiles, which are headed by the Job Profile, form the basis of a DB2 Automation Tool task. We can submit this task manually or schedule it by using the DB2 administration task.
Step 5: Execute the jobs in a predefined maintenance window
Today, a typical maintenance strategy has pre-defined jobs in a job scheduler. These jobs are run in maintenance widows weekly, monthly and quarterly. With the Autonomics Framework, you can leverage your own batch scheduler for spawning evaluation jobs as well as starting the Autonomics Director procedure at any time during your maintenance window.
After following these steps, transforming your passive into an active autonomic environment, the corrective actions are taken automatically by the system – i.e. monitoring and analyzing the related metrics to pro-actively make recommendations and even execute them. These are tasks typically done by a DBA. With these automated basic administration tasks you give DBAs freedom to work on higher business value tasks. And more important they do not rely on old, homegrown processes which are difficult to maintain and keep up with new DB2 versions.
And how about you - have you already moved from passive to active strategy in your environment? What benefits have you seen? Tell us which experiences you gained regarding the change process.