May 2024: APAR OA66223
2024 Q1 maintenance.
OMEGAMON Data Broker
- New version of forwarder protocol
- The OMEGAMON Data Broker forwarder protocol is a
bespoke TCP/IP-based protocol for forwarding data to OMEGAMON Data Connect.
This APAR introduces version 2 of the forwarder protocol. The original protocol is now known as version 1. By default, OMEGAMON Data Broker now uses version 2 of the protocol.
Important: Upgrade all of your instances of OMEGAMON Data Broker and OMEGAMON Data Connect at the same time. Earlier APAR levels of OMEGAMON Data Connect cannot receive data that is sent using protocol version 2. For example, if you upgrade OMEGAMON Data Broker but not OMEGAMON Data Connect, then OMEGAMON Data Connect will no longer receive data.Version 2 includes the following enhancements:
- Improved performance
- OMEGAMON Data Broker now sends a batch of records in
each message to OMEGAMON Data Connect.
Previously, using version 1, OMEGAMON Data Broker sent only a single record in each message.
- Avoiding data loss under certain conditions
- OMEGAMON Data Broker no longer discards unsent
records in the forwarder queue if the connection to OMEGAMON Data Connect is lost.
Previously, using version 1, if OMEGAMON Data Broker lost connection to OMEGAMON Data Connect, then OMEGAMON Data Broker discarded unsent records in the forwarder queue.
Now, if OMEGAMON Data Connect reconnects before the forwarder queue fills, then OMEGAMON Data Broker does not lose data.
Hence, depending on the data rate and the time it takes to restart OMEGAMON Data Connect, you can now restart OMEGAMON Data Connect without causing data loss in OMEGAMON Data Broker.
Version 2 introduces two new optional OMEGAMON Data Broker configuration parameters:
PROTOCOL_VERSION
andMAX_BATCH_SIZE
. Typically, you don't need to be aware of these parameters. - Store parameters for queue capacity and cells are now optional
- The following parameters are now optional:
KAY.CIDB.STORE.<store_id>.QUEUE.CAPACITY
KAY.CIDB.STORE.<store_id>.CELL.<cell_id>.SIZE
KAY.CIDB.STORE.<store_id>.CELL.<cell_id>.CAPACITY
These parameters expose implementation details that you typically don't need to know. Unless you have site-specific reasons to specify non-default values, you should remove these parameters from your configuration members.
- Reusable sink parameters
- You can now define reusable sink parameters separately from forwarders. Multiple
forwarders can refer to the same set of reusable sink parameters. Reusable sink parameters make it
easier to forward data from different sources to the same sink (the same instance of OMEGAMON Data Connect). For
example:
* Stores KAY.CIDB.STORE.STORE1.NAME=OMEGAMON KAY.CIDB.STORE.STORE2.NAME=SOURCEB * New: reusable sink parameters KAY.CIDB.SINK.SINK1.HOST=analytics1.example.com KAY.CIDB.SINK.SINK1.PORT=15351 KAY.CIDB.SINK.SINK1.SECURITY=TLSv1.2 KAY.CIDB.SINK.SINK1.FIPS=ON KAY.CIDB.SINK.SINK1.KEYRING=KAYSIS01/KAYring * Forwarders that refer to reusable sink parameters KAY.CIDB.FWD.FWD1.SOURCE_STORE=OMEGAMON KAY.CIDB.FWD.FWD1.SINK=SINK1 KAY.CIDB.FWD.FWD2.SOURCE_STORE=SOURCEB KAY.CIDB.FWD.FWD2.SINK=SINK1
Previously, to define multiple forwarders that use the same sink, you had to duplicate the sink parameters for each forwarder. For example (for conciseness, this example omits store queue capacity and cell parameters, which are now optional):
* Stores KAY.CIDB.STORE.STORE1.NAME=OMEGAMON KAY.CIDB.STORE.STORE2.NAME=SOURCEB * Forwarders (with duplicate sink parameters) KAY.CIDB.FWD.FWD1.SOURCE_STORE=OMEGAMON KAY.CIDB.FWD.FWD1.SINK_HOST=analytics1.example.com KAY.CIDB.FWD.FWD1.SINK_PORT=15351 KAY.CIDB.FWD.FWD1.SECURITY=TLSv1.2 KAY.CIDB.FWD.FWD1.FIPS=ON KAY.CIDB.FWD.FWD1.KEYRING=KAYSIS01/KAYring KAY.CIDB.FWD.FWD2.SOURCE_STORE=SOURCEB KAY.CIDB.FWD.FWD2.SINK_HOST=analytics1.example.com KAY.CIDB.FWD.FWD2.SINK_PORT=15351 KAY.CIDB.FWD.FWD2.SECURITY=TLSv1.2 KAY.CIDB.FWD.FWD2.FIPS=ON KAY.CIDB.FWD.FWD2.KEYRING=KAYSIS01/KAYring
Sink parameters that are specified with a forwarder are still supported, and are now known as forwarder-scope sink parameters.
- Improved validation of configuration parameters
- OMEGAMON Data Broker now validates more configuration parameters at startup. Warning message KAYB0023W now reports more bad parameter values.
OMEGAMON Data Connect
- Support for new version of OMEGAMON Data Broker forwarder protocol
- OMEGAMON Data Connect supports the new version 2 of
the OMEGAMON Data Broker forwarder protocol.
Instances of OMEGAMON Data Connect that have been upgraded to this APAR level can receive data using either protocol version 1 or 2. That is, upgraded instances of OMEGAMON Data Connect can receive data from earlier APAR levels of OMEGAMON Data Broker.
For more details, see the corresponding change for OMEGAMON Data Broker.
- Improved validation of configuration parameters
- Previously, OMEGAMON Data Connect silently ignored unknown
parameters in the configuration file. That behavior included ignoring parameters that were known in
some contexts, but unknown in the context they were specified in the file. The configuration file is
a YAML document, so the indentation of a parameter determines its context. Incorrect indenting could
cause issues that were difficult to troubleshoot.
For example, suppose you specify the following parameters with the intention of enabling output to STDOUT:
connect: output: stdout: enabled: true # Incorrectly indented
These parameters won't have the intended effect, because the
enabled
key is unknown at the same indent level asstdout
. Instead,enabled
should be indented as a child ofstdout
. Previously, OMEGAMON Data Connect silently ignoredenabled: true
in this context. The result was that output to STDOUT was not enabled.Now, if the configuration file contains an unknown parameter, OMEGAMON Data Connect fails at startup.
Attention: If your configuration file contains errors that were previously silently ignored, this is a breaking change. You must edit the configuration file and fix the issue. - Configuration validator
- The OMEGAMON Data Provider configuration validator is a new
command-line tool that you can use to validate the two YAML-format OMEGAMON Data Provider configuration files:
- The OMEGAMON Data Connect configuration file
- If you are using OMEGAMON monitoring agents as a data source: the OMEGAMON Data Provider collection configuration member, RKANPARU(KAYOPEN)
After editing these configuration files, you can run the configuration validator as a
preflight
check before using the files. - Sample shell script z/OS® UNIX file now encoded using EBCDIC code page 1047
- You no longer need to activate automatic text conversion to run the sample shell script to start
OMEGAMON Data Connect on z/OS
UNIX.
Previously, the script was supplied on z/OS UNIX tagged as a text file that is uniformly encoded in code set ISO8859-1. While this encoding and tagging is a common practice for text files on z/OS UNIX, by default z/OS UNIX interprets shell scripts using EBCDIC encoding. To run the script on z/OS UNIX, you had to activate automatic text conversion of tagged UNIX file system files by setting the
_BPXK_AUTOCVT
environment variable toON
. Otherwise, when you ran the script, z/OS UNIX attempted to interpret the script using EBCDIC encoding and reported the error message FSUM7332 syntax error.
Documentation-only changes
- Configuring OMEGAMON Data Broker
- OMEGAMON Data Broker is a plug-in for the Zowe cross-memory server. You can configure OMEGAMON Data Broker using one of the following servers:
- The server that is supplied with OMEGAMON Data Provider
- An existing server in a separate Zowe installation
This documentation now recommends the server that is supplied with OMEGAMON Data Provider as the best and easiest choice for most users, with reasons.
Previously, a single procedure covered both options, with conditional steps for each option. Now, each option has its own simpler procedure.