As an essential part of IBM Z pervasive encryption, z/OS Encryption Readiness Technology (zERT) has earned a lot of attention after it made its debut with z/OS V2R3 in September, 2017.
Having launched zERT discovery and zERT aggregation, the z/OS Communications Server team released the latest functional enhancement of the zERT family, the IBM zERT Network Analyzer, in December, 2018. The IBM zERT Network Analyzer is a new z/OSMF plugin that provides a web-based graphical user interface that z/OS network security administrators can use to analyze and report on data reported in zERT Summary records.
To use IBM zERT Network Analyzer, please be aware of the following dependencies:
You must have installed z/OSMF V2R3 APARs PH04391 and PH00712 to use IBM zERT Network Analyzer.
The IBM zERT Network Analyzer task requires Db2 11 for z/OS and above.
To get a quick start with IBM zERT Network Analyzer, check out the IBM zERT Network Analyzer overview demo video:
You can also refer to the IBM zERT Network Analyzer tutorial to get a detailed idea of how the IBM zERT Network Analyzer can help you assess the cryptographic protection of your z/OS network traffic.
For more details about the IBM zERT Network Analyzer function, see the related topic in IBM Knowledge Center.
The Configuration Assistant for z/OS Communications Server (Network Configuration Assistant) provides a guided interface to help you create and manage TCP/IP profiles, and simplify configuration of TCP/IP policy-based networking functions. You access the Network Configuration Assistant under the “Configuration” header in the z/OSMF navigation pane.
We have created a brief survey to get feedback on the Network Configuration Assistant. You don’t need to be using the Network Configuration Assistant to take the survey.
This short survey should only take 2 - 3 minutes. Your anonymous responses will only be used to help us understand and enhance your experience with the Network Configuration Assistant.
Click the survey link or scan the QR code to provide feedback:
The 2019 SHARE conference is March 10 - 15 in the Phoenix Convention Center, Phoenix, AZ. As always, there will be a good selection of content focused on z/OS Communications Server, Network Configuration Assistant, and ISPF, including the following sessions from our Enterprise Networking Solutions team in Research Triangle Park, NC.
z/OS Encryption Readiness Technology (zERT) is an exciting new feature of z/OS V2R3 Communications Server and a core capability of IBM Z pervasive encryption. zERT will assist you in discovering and analyzing the status of the network cryptographic protection of your z/OS TCP and Enterprise Extender workloads.
IBM zERT Network Analyzer, a z/OSMF plugin, is now available for you to easily query and analyze the data that zERT provides.
Providing the data you need to build a complete picture of your z/OS cryptographic network protection posture
Live demo of IBM zERT network analyzer:
A web-based graphical user interface that makes it easy to analyze the data that zERT provides
In this webcast, Chris will focus on the new capabilities provided by zERT. A previous zERT session presented at the SHARE conference in St. Louis was identified as an outstanding IBM session and the winner of a Best Session Award.
No organization on the planet can escape the effects of today’s digital transformation. Ensuring quality in every interaction and transaction builds trust that is at the core of every relationship with clients and partners.
The IBM z14 (z14) was designed as the infrastructure to trust in the digital economy. It delivers function and capabilities to meet the demands for new services and better customer experiences, while securing the growing amounts of data and complying with increasingly intricate regulations.
Now IBM extends the z14 family with a new single-frame model able to process over 850 million fully encrypted transactions per day in the space of two floor tiles. Now, businesses of all sizes can create a low-cost, secure cloud infrastructure and capitalize on new opportunities through trusted digital experiences.
What is new?
The IBM z14 with GA2 code takes your workloads to a new level by increasing your networking bandwidth to 25 GbE for both OSA and RoCE. The higher bandwidth creates opportunities to move more data with the same number of adapters or exploit fewer adapters by consolidating lower bandwidth adapters.
A summary of the z14 GA2 25 GbE features that are introduced are:
OSA-Express7S 25 GbE providing OSD CHPID (QDIO mode) support as a single 25 GbE port per feature. With this next generation of OSA-Express and when your z/OS Communications Server interface is configured in QDIO mode, all existing OSA (OSD) features are preserved, but now provided with a much higher bandwidth for your business critical TCP/IP workloads. (OSA-Express7S 25 GbE is planned for availability in April, 2019.)
25 GbE RoCE Express2 is provided for increasing the bandwidth of your RoCE-related workloads.
A quick review of the 10 GbE RoCE Express2 feature that can replace the 10 GbE RoCE Express feature shows that the 10 GbE RoCE Express2 continues to enable IBM Z server-to-server Shared Memory Communications (SMC-R). SMC-R is designed to take advantage of high speed protocols and direct memory placement of data for faster communications versus consuming large amounts of TCP/IP resources. The RoCE Express2 feature provides increased virtualization (sharing capability) allowing RoCE to be extended to more workloads by allowing additional virtual functions (VFs) per physical port.
The new 25 GbE RoCE Express2 with z14 GA2 code with the increased bandwidth complements the option for better adapter sharing. This could provide flexibility for clients to consolidate adapters or increase overall speed, while still seeing the high reliability and performance established on prior adapters.
The z14 continues to support Shared Memory Communications - Direct Memory Access (SMC-D) to improve direct memory communications between logical partitions on a single server. SMC-D optimizes z/OS for improved performance in “within-the-box” communications versus standard TCP/IP HiperSockets or Open Systems Adapter. SMC-D is compatible with SMC-R for all RoCE features.
As we discussed in a previous article, data breaches continue to be costlier and result in more consumer records being lost or stolen in 2018. Now let’s see how pervasive encryption makes a difference.
Extensive use of encryption is one of the most powerful ways to help reduce the risks of a data breach and help meet complex compliance mandates. Implementing encryption can be complex. Organizations always struggle with three key questions:
What data should be encrypted?
Where should encryption occur?
Who is responsible for encryption?
IBM Z pervasive encryption marks a paradigm shift for security. Pervasive encryption provides a transparent and consumable approach to enable extensive encryption of data in flight and at rest to simplify and reduce the costs associated with protecting data and achieving compliance mandates.
To make this happen, IBM Z delivered several new capabilities through tight full-stack platform integration, in the hardware, OS, and middleware.
As highlighted in this diagram, to protect in-flight network data and APIs, IBM z14 can encrypt incoming and outgoing network connections for true end-to-end data protection.
In general, z/OS supports three robust end-to-end security protocols:
Transport Layer Security (TLS, including SSL) via System SSL and Java Secure Sockets Extension (JSSE)
IPsec, which is built into z/OS Communications Server
Secure Shell (SSH) via z/OS OpenSSH
Each protocol has its place and is suited for different types of traffic. Which protocol you select to protect each type of application traffic depends on a number of different factors. In most cases, you have a choice of protocols for any given application type. (See the attached table for a comparison of key protocols.)
Meanwhile, you need a tool to identify z/OS network traffic and protection. z/OS Encryption Readiness Technology (zERT), a core capability of pervasive encryption, does exactly that.
zERT provides intelligent network security discovery and reporting capabilities by monitoring TCP and Enterprise Extender traffic for TLS/SSL, IPsec and SSH protection, as well as cleartext. It also writes protection information to new SMF 119 records. Moreover, zERT Network Analyzer, a new web-based interface that is planned to be available in the future, will help you determine which z/OS TCP and Enterprise Extender traffic is or isn’t protected according to specific query criteria.
z/OS Encryption Readiness Technology (zERT), a core capability of IBM Z pervasive encryption, is an important feature of z/OS V2R3 Communications Server. This exciting new technology will assist you in analyzing the status of the z/OS network cryptographic protection of your TCP and Enterprise Extender workloads.
Your answers to this brief survey will be very helpful for us, the z/OS Communications Server team, to understand your needs and plans in identifying where and how network traffic is protected.
This short survey should only take 2 - 3 minutes. Your anonymous responses will only be used to help us understand and enhance your experience with z/OS Communications Server.
Click the survey link or scan the QR code to provide feedback:
A data breach is every company's waking nightmare. You may wonder about the actual cost for your company.
IBM Security and Ponemon Institute just released the 2018 Cost of a Data Breach Study. Interviews were conducted with more than 2,200 IT, data protection, and compliance professionals from 477 companies that have experienced a data breach over the past 12 months. According to the findings, data breaches continue to be costlier and result in more consumer records being lost or stolen, year after year.
The Global Cost of a Data Breach Is Up in 2018
In this year’s study, the average total cost of a data breach was $3.86 million, and it took companies 196 days, on average, to detect a breach. Moreover, the likelihood of a recurring material breach over the next two years was 27.9%.
The Bigger the Breach, the Higher the Cost
In this year’s study, the cost ranged from $2.1 million for incidents with less than 10,000 compromised records to $5.7 million for incidents with more than 50,000 compromised records. Each year, the findings show a consistent relationship between cost and size of the data breach.
Want to see how much a breach would cost your company?
Check out this survey to provide your feedback on zERT.
z/OS Encryption Readiness Technology (zERT), a core capability of IBM Z pervasive encryption, is an important feature of z/OS V2R3 Communications Server.
zERT provides intelligent network security discovery and reporting capabilities by monitoring TCP and Enterprise Extender traffic for TLS/SSL, IPsec and SSH protection, as well as cleartext. It also writes information about the state of that protection to new SMF 119 records. Moreover, IBM zERT Network Analyzer, a new web-based interface on z/OSMF, available since December, 2018, helps you determine which z/OS TCP and Enterprise Extender traffic is or isn’t protected according to specific query criteria.
The following mainframe events include sessions about zERT:
zERT aggregation provides an alternative SMF view of the collected security session data in the form of SMF 119 zERT Summary (subtype 12) records that summarize the repeated use of security sessions by many application connections over time. zERT Summary records are written at the end of each SMF interval. Compared to zERT discovery alone, zERT aggregation can significantly reduce the volume of SMF records while still providing the critical security information.
zERT discovery discovers the network encryption attributes for each TCP and Enterprise Extender connection by positioning the z/OS TCP/IP stack as a central collection and reporting point for the cryptographic protection attributes for TLS, SSL, SSH, and IPSec security sessions.
Getting a grip on your z/OS network encryption!
Time: 2/26/2019 11:00 AM EST
Speaker: Chris Meyer
Duration: 60 minutes
Learn more details and register to watch the webinar recording here.
The following technical session presentations will provide more details on zERT:
The z/OS Communications Server SMTP (CSSMTP) client was introduced in z/OS V1R11 to provide a modern, RFC-compliant mail client that would perform better and provide superior client function to the existing SMTPD program. Unlike SMTPD, CSSMTP supports IPv6 and can have its communications with a mail server secured with application-transparent TLS (AT-TLS).
In February of 2014, IBM issued a statement of direction (SoD) that stated that support for SMTPD and sendmail would be removed in a future release. In July of 2015, a subsequent SoD further elaborated on that direction by stating that z/OS V2R2 would be the last release to include SMTPD and sendmail.
z/OS V2R3, which became generally available in September of 2017, fulfills that SoD by no longer including SMTPD or sendmail. With V2R3, CSSMTP is the only mail client provided with z/OS. (z/OS no longer provides a mail server capability in z/OS V2R3.)
There was one significant capability missing from CSSMTP as compared to SMTPD: support for multibyte character sets (MBCS). MBCS translation, and in particular, double-byte character set (DBCS) support, is a capability that was supported by SMTPD and is used by many customers in some geographies. As such, the lack of DBCS support in CSSMTP is a hindrance to CSSMTP acceptance in those geographies, and therefore an inhibitor to customer migrations to z/OS V2R3.
The good news is that this missing DBCS support is now being provided through new-function APAR PI93278 for z/OS V2R1, V2R2, and V2R3. Unlike SMTPD, which depended on translation tables, CSSMTP's DBCS support is enabled via specification of a multi-byte code page through a new MBCHARSET operand in the CSSMTP configuration file (along with an MBCS switch to control whether MBCS capability is to be enabled at all). A table in the IP Configuration Guide provides suggestions as to which code page to specify on the MBCHARSET parameter.
An Additional Enhancement for SBCS Users
APAR PI93278 is labeled "code page enhancements' because there are actually two significant enhancements provided via this APAR. In addition to MBCS support, the APAR also provides improved handling of single-byte character set (SBCS) special characters in the mail subject line. Previously, some special characters, such as the euro symbol (€), were not translated correctly. This APAR is also intended to address that issue.
The PTFs for APAR PI93278 are:
For those exploiting the MBCS support, you should also apply the PTF for APAR OA55727 from Unicode Services:
The June and July 2018 enhancements of z/OS Communications Sever roll out exciting new features for your business. Read on for a quick introduction to these new features and links to resources offering more information.
Scalability and performance enhancement
IWQ support for IPSec
z/OS V2R3 Communications Server, with TCP/IP APAR PI77649, is enhanced to support inbound workload queueing for IPSec workloads for OSA-Express in QDIO mode.
Note: This new function is also available in V2R2 via TCP/IP APAR PI77649 and SNA APAR OA52275.
Incompatibilities: This function does not support IPAQENET interfaces that are defined by using the DEVICE, LINK, and HOME statements. Convert your IPAQENET definitions to use the INTERFACE statement to enable this support.
This function is limited to OSA-Express6S Ethernet features or later in QDIO mode running on IBM z14.
This function is supported only for interfaces that are configured to use a virtual MAC (VMAC) address.
z/OS V2R3 Communications Server, with APAR PI93278, is enhanced to support multi-byte character sets with the Communications Server SMTP (CSSMTP) application. This enhancement allows migration from SMTPD to CSSMTP for customers that use multi-byte character set code pages, and provides improved code page support for characters in the mail subject line.
Note: This new function is also available in V2R1 and V2R2 via the same APAR.
The summer 2018 SHARE conference is August 12 - 17 in the America's Center Convention Complex, St. Louis, MO. As always, there will be a good selection of content focused on z/OS Communications Server, Network Configuration Assistant, Multi-site Workload Lifeline, and ISPF, including the following sessions from our Enterprise Networking Solutions team in Research Triangle Park, NC.
Our last few blog entries have described the new z/OS Encryption Readiness Technology (zERT) features of z/OS Communications Server V2R3. In this edition, let’s recap a few of the points and cover a few important considerations to keep in mind as you plan your zERT deployment.
1. zERT can generate very large volumes of zERT Connection Detail (SMF 119 subtype 11) records, depending on the number of connections supported by your z/OS system.
Please plan accordingly.
Consider only capturing zERT Summary (SMF 119 subtype 12) records on a regular basis and only capture subtype 11s for limited times when investigating specific traffic.
2. zERT monitors TCP and Enterprise Extender traffic. All other IP protocols are unmonitored.
3. zERT monitors traffic that terminates at the local TCP/IP stack. It does not monitor routed traffic
4. zERT does not store or record the values of secret keys, initialization vectors, or any other secret values that are negotiated or derived during cryptographic protocol handshakes.
5. Regardless of the prior point, the zERT data that is recorded provides a rather complete picture of the z/OS system’s network cryptographic protection profile. As such, you should take appropriate steps to protect the recorded SMF data as well as access to the SYSTCPER and SYSTCPES NMI services.
6. zERT only monitors connections that are established after zERT discovery is enabled (or re-enabled).
If you disable and later re-enable zERT, it will not monitor any of the connections that existed before re-enabling.
To ensure the most complete monitoring, enable zERT discovery in your TCP/IP profile.
Remember that you can enable and disable the recording of zERT discovery and zERT aggregation data (the zERT Connection Detail and zERT Summary records) via the SMFCONFIG and NETMONITOR parameters independently of zERT monitoring.
7. TCP traffic protected by cryptographic protocol providers that are not enabled for zERT (OpenSSL, other SSH implementations, etc.) will only be reported through stream observation, which has the following limitations:
Stream observation can only report the initial TLS/SSL or SSH handshake as long as it is the first thing to flow over the TCP application connection after that connection is established. It has no visibility to rehandshakes or early termination of security sessions.
zERT has no visibility to attributes that are negotiated during the initial handshake using encrypted messages.
8. There are a small number of System SSL applications that cannot be monitored and are therefore reported as being unprotected. These are applications that:
Send or receive application-specific data before initiating the TLS/SSL session.
Do not use the SIOCSHSNOTIFY ioctl.
9. In certain mixed-release sysplex environments, some IPSec-related attributes will not be available for reporting.
zERT provides the raw data for unprecedented insight to the cryptographic protection of your z/OS network traffic. So the next time you are asked “which traffic is protected” or “how is our traffic protected,” look to zERT to provide the answers!
We have already introduced z/OS Encryption Readiness Technology (zERT), which monitors a wide variety of cryptographic protection attributes for TCP and Enterprise Extender (EE) traffic on z/OS, and provided some details of zERT discovery and zERT aggregation. Now let's clearly define some of the important terms that you’ll see used with zERT.
Cryptographic Protocol Provider (CPP):A z/OS-resident component that processes a specific cryptographic network security protocol (i.e., TLS/SSL, IPSec or SSH). There are a few categories of CPPs:
– IBM zERT-enabled CPPs: z/OS System SSL, z/OS OpenSSH and the IPSec support in the z/OS TCP/IP stack
– IBM non-zERT-enabled CPPs: Java Secure Sockets Extension (JSSE). As of this writing, z/OS JSSE is not zERT-enabled
– 3rd party non-zERT-enabled: Tectia SSH for z/OS, OpenSSL (ported to z/OS), etc.
Protection state:The cumulative state of cryptographic protection of a connection. There are several possible combinations here:
– No cryptographic protection (connection is in cleartext mode)
– Protection from a single cryptographic protocol (the most common case)
– Protection from multiple cryptographic protocols (for example, a TCP connection protected by both TLS and IPSec)
The key point here is that a single TCP connection can be protected by zero or more cryptographic protocols. For Enterprise Extender, which is based on UDP, the only supported cryptographic protocol is IPSec.
Application connection:A sockets-based connection between two application programs. No security is implied or provided – it’s just a cleartext path over which two programs communicate.
Security session:The application (by a CPP) of an agreed-to set of security attributes (as defined by a cryptographic security protocol) to one or more application connections between the same client and server. Examples are TLS/SSL sessions, IPSec tunnels, and SSH sessions.
Here is an example using TLS to help you better understand the differences between an Application connection and a Security session.
Client application establishes a TCP connection with the server.
Client application initiates a TLS handshake which authenticates the server (and, optionally, client) and negotiates a cipher suite and keys to be used to protect the data. Upon successful completion of the handshake, a secure TLS session exists between the communication partners to protect the TCP connection.
Data flows over the TCP connection under protection of the TLS session using the cryptographic algorithms and keys negotiated during the handshake.
A firm understanding of these terms will make it easier to understand and to use zERT discovery and zERT aggregation.
As we discussed in a previous article, zERT discovery gives z/OS network administrators a way to effectively monitor z/OS network security status. However, workloads that consist of large numbers of frequent short-lived connections could generate huge volumes of zERT subtype 11 records. Although some measures are already taken in zERT discovery to reduce the number of records, these measures may be insufficient in environments that manage thousands of connections per hour or minute.
zERT aggregation, available with new function APAR PI83362, is designed to provide the same level of cryptographic detail with much lower SMF volume than zERT discovery can generate.
zERT aggregation summarizes the repetitive use of security sessions over time. Security sessions are summarized from the server’s perspective (based on server IP address, server port, and client IP address), regardless of whether z/OS is the client or the server. For Enterprise Extender traffic, they are always summarized from the local z/OS peer’s perspective. Summaries are written at the end of each SMF interval through new SMF 119 zERT summary (subtype 12) records which contain:
Connection attributes (Server IP addr, server port, client IP addr, transport protocol)
Significant security attributes (those that materially contribute to the strength of the cryptographic protection)
Statistics (connection counts, byte counts, etc.)
With aggregation, the data recorded across a large number of SMF 119 subtype 11 records can be greatly condensed into a small set of SMF 119 subtype 12 records.
zERT aggregation configuration
Like zERT discovery, aggregation is enabled independently of the recording destinations:
A new GLOBALCONFIG ZERT sub-parameter enables/disables aggregation: GLOBALCONFIG ZERT AGGregation | NOAGGregation (the default is NOAGGREGATION)
A new SMFCONFIG parameter to configure writing of SMF 119 subtype 12 records to SMF: SMFCONFIG ZERTSUMmary | NOZERTSUMmary (default is NOZERTSUMMARY)
A new NETMONITOR parameter to configure writing of SMF 119 subtype 12 records to the SYSTCPES realtime NMI service: NETMONITOR ZERTSUMmary | NOZERTSUMmary (default is NOZERTSUMMARY)
All parameters can be dynamically enabled or disabled and are exposed in the z/OSMF-based Configuration Assistant for Communications Server under the TCP/IP profile perspective.
The same configuration reporting interfaces that were updated for discovery are also updated for aggregation.
Realtime zERT aggregation network monitoring service
The new SYSTCPES NMI service makes zERT 119 SMF zERT Summary (subtype 12) records available to network management applications as they are generated. Like the SYSTCPER service for zERT discovery, SYSTCPES uses the same programming model as SYSTCPCN (TCP connection service). For more details, see z/OS Communications Server: IP Programmer’s Guide in the IBM Knowledge Center.
SMF 119 subtype 12 "zERT Summary" record
Subtype 12 records are written at the end of each SMF interval. Since they are associated with security sessions and not individual application connections, they contain either zero or one cryptographic protocol-specific section (zero for cleartext sessions, one for TLS/SSL, IPSec or SSH sessions). Contrast this with the per-connection subtype 11 records which can contain zero or more cryptographic protocol-specific sections (zero for cleartext connections, one or more for connections that are protected by some combination of TLS/SSL, IPSec and SSH).
The general layout of the subtype 12 record is illustrated in the graphic at the top of this blog entry.
With zERT discovery and aggregation in place, you now have the option of collecting the lower-volume zERT Summary records on an ongoing basis to maintain a constant watch on your z/OS network protection posture. Then, when you when you need to do more in-depth investigation of specific traffic patterns, you can enable the recording of the per-connection zERT Connection Detail records.