PJ46608: Real-time runtime metrics collection multi-JVM monitoring and multi-processor support.

APAR status

• Closed as new function.

Error description

• See Problem Summary.
  

Local fix

• NA
  

Problem summary

• APAR NUMBER:  PJ46608
  PRODUCT:  z/TPF
  FUNCTIONAL AREA:  NAME VALUE PAIR COLLECTION
  SHIPPED IN YEAR:  2022
  
  ABSTRACT:
  Real-time runtime metrics collection multi-JVM monitoring and
  multi-processor support.
  
  PACKAGE CONTENTS:
  Source Segments:
  (C) base/include/tpf/crrn.hpp
  (C) base/include/tpf/crrs_send.hpp
  (C) base/macro/irtmc.mac
  (C) base/rt/crrn_send_preformatted_data.cpp
  (C) base/rt/rt_rtmc_start.cpp
  (C) base/tpf-fdes/rtmc_preformatted_data_header.gen.dfdl.xsd
  (C) base/tpfrtmc/bin/tpfrtmc.tar.gz
  
  Object Only Binaries:
  None.
  
  BINARIES TO BUILD: YES
  Configuration Independent Binaries:
  (C) base/lib/libCRRN.so
  (C) base/load/CRRN.so
  (C) base/obj/crrn_send_preformatted_data.o
  (C) base/obj/rt_rtmc_start.o
  (C) base/lib/libCNVI.so
  (C) base/lib/libCRRI.so
  (C) base/lib/libCRRS.so
  (C) base/load/CNVI.so
  (C) base/load/CRRI.so
  (C) base/load/CRRS.so
  (C) base/load/CRRT.so
  (C) base/load/CRRW.so
  (C) base/load/CTLG.so
  (C) base/obj/cnvi.o
  (C) base/obj/ctlg.o
  (C) base/obj/rtmc_globals.o
  (C) base/obj/rtmc_utils.o
  (C) base/obj/rt_rtmc_display.o
  (C) base/obj/rt_rtmc_display_statistics.o
  (C) base/obj/rt_rtmc_mat.o
  (C) base/obj/rt_rtmc_modify.o
  (C) base/obj/rt_rtmc_stop.o
  (C) base/obj/crrt.o
  (C) base/obj/crrt_defs.o
  (C) base/obj/crrw.o
  (C) base/obj/crrs_send.o
  
  Configuration Dependent Binaries:
  (C) <sys>/load/CPS0.so
  (C) <sys>/obj/ccctin.o
  (C) <sys>/obj/ccenbk.o
  (C) <sys>/obj/ccnucl.o
  (C) <sys>/obj/ccstor.o
  (C) <sys>/obj/cnvp.o
  (C) <sys>/stdlib/libCTAL.so
  (C) <sys>/stdload/CTAL.so
  
  COMMENTS:
  The initial release of real-time runtime metrics collection
  (PJ45657) and the z/TPF real-time insights dashboard starter
  kit could only process, analyze and display data for a single
  z/TPF processor.
  
  PJ46275 introduced the ability for real-time runtime metrics
  collection to monitor java virtual machines (JVM) running on
  z/TPF. However, the tpfrtmc offline utility could only monitor
  JVM data from a single z/TPF system.
  

Problem conclusion

• SOLUTION:
  The z/TPF real-time runtime metrics collection system wide JVM
  monitor support is enhanced to allow a single instance of the
  tpfrtmc offline utility to process data from up to 10 z/TPF
  processors simultaneously. The z/TPF real-time insights
  dashboard starter kit was enhanced to allow you to view results
  from multiple z/TPF processors.
  
  Also, the z/TPF real-time insights dashboard starter kit has
  been updated to:
  * Allow a user to select a data source. This feature allows the
  dashboard to be pointed to a different MariaDB database. Each
  data source may represent a different z/TPF complex, test
  environment, geography, etc.
  * Allow a user to select a z/TPF processor from the selected
  data source. The processor is represented by the: system name
  (as shown by ZSTAT GPU, LPAR or VM Guest name), complex,
  processor id, IP address, environment, and central electronic
  complex (CEC).
  * Allow a user to filer system names by CEC.
  * Modeled_CPU calculations were enhanced in accordance with
  PJ46610 to accomodate Fenced I-streams, HiperDispatch, and
  I-stream boost scenarios.
  * 18 dashboards were condensed to 2 dashboards for better
  usability and performance.
  * Dashboard folders were reorganized and simplified for better
  usability.
  * JVM Dashboards now have better response time for large time
  frames.
  * JVM Dashboards now have better support for JVMs on multiple
  subsystems.
  * Name-value pair statistics dashboard better protects against
  large poorly performing time frames.
  * MariaDB write database was moved to a volume on the
  filesystem so that data persists across MariaDB Docker builds.
  * Various other exceptions and issues were fixed.
  
  At this time, a single instance of the tpfrtmc offline utility
  can only accept continuous data collection (CDC) and name-value
  pair collection data from a single z/TPF processor.
  
  For details about how to install and use the starter kit, see
  the z/TPF real-time insights dashboard starter kit download
  page.
  
  COREQS: NO
  None.
  
  MIGRATION CONSIDERATIONS: YES
  Application programming interface (API) changes:
  In the z/TPF real-time insights dashboard starter kit, all
  Python classes in the ZRTMC Analyzer ending in V10 are changed
  so that the V10 is removed. Example CdcDataV10 -> CdcData.
  Customers should replace all "V10" with "" in their user code.
  
  Database changes:
  We are changing the starter kit to have mariadb write database
  data to a volume on the filesystem so that data persists across
  MariaDB Docker builds. Customers can keep their existing data
  by running the following commands BEFORE deploying the new
  starter kit. This is only necessary for customers upgrading
  from a starter kit prior to PJ46608 rtRTMC multi-JVM and
  dashboard data source/system selector:
  * cd tpf_data_sci/Docker
  * mkdir -p ./tpf_mariadb_docker_files/db
  * docker cp tpf-mariadb:/var/lib/mysql/.
  tpf_mariadb_docker_files/db
  
  Here is the list of changes to the MariaDB database for the
  starter kit processing of real-time runtime metrics collection.
  
  New tables:
  version_history (The number of rows in this table will be
  negligible)
  - version_id, INT, 4 bytes
  - version, VARCHAR(16), 17 bytes
  - valid_from, DATETIME, 8 bytes
  - valid_to, DATETIME, 8 bytes
  - major, INT, 4 bytes
  - minor, INT, 4 bytes
  - PRIMARY KEY (version_id)
  tpf_systems (The number of rows in this table will be
  negligible)
  - system_name, VARCHAR(8), 9 bytes
  - complex_name, VARCHAR(8), 9 bytes
  - processor_id, VARCHAR(1), 2 bytes
  - ip_address VARCHAR(15), 16 bytes
  - cec_name, VARCHAR(16), 17 bytes
  - environment, VARCHAR(10), 11 bytes
  - id, SMALLINT, 2 bytes
  - PRIMARY KEY (id)
  - UNIQUE KEY (processor_id, system_name, ip_address,
  complex_name, cec_name)
  tpf_jams (The number of rows in this table will be negligible)
  - system_id, SMALLINT, 2 bytes
  - subsystem, VARCHAR(4), 5 bytes
  - jam, VARCHAR(32), 33 bytes
  - id, MEDIUMINT, 3 bytes
  - PRIMARY KEY (id)
  - UNIQUE KEY (system_id, subsystem, jam)
  jvm_operations (The number of rows in this table will be
  negligible)
  - operation, VARCHAR(128), 129 bytes
  - id, MEDIUMINT, 3 bytes
  - PRIMARY KEY (id)
  - UNIQUE KEY (operation)
  
  
  Updates to existing tables:
  system_metrics_table
  - ADD COLUMN system_id, SMALLINT, +2 bytes
  - ADD COLUMN istream_cap, INT, +4 bytes
  - DROP COLUMN active_istreams, INT -4 bytes
  - DROP PRIMARY KEY (time)
  - ADD PRIMARY KEY (system_id, time)
  - ADD INDEX (time)
  system_tcpip_table
  - ADD COLUMN system_id, SMALLINT, +2 bytes
  - DROP INDEX (time,app_name)
  - ADD PRIMARY KEY (system_id, app_name, time)
  - ADD INDEX (time)
  msg_types_table
  - ADD COLUMN system_id, SMALLINT, +2 bytes
  - DROP INDEX (time,msg_type)
  - ADD PRIMARY KEY (system_id, msg_type, time)
  - ADD INDEX (time)
  horizontal_nvps_table
  - ADD COLUMN system_id, SMALLINT, +2 bytes
  - DROP INDEX (time,nvps)
  - ADD PRIMARY KEY (system_id, nvps, time)
  - ADD INDEX (time)
  code_packages_table
  - ADD COLUMN system_id, SMALLINT, +2 bytes
  - DROP INDEX (time,code_package)
  - ADD PRIMARY KEY (system_id, code_package, time)
  - ADD INDEX (time)
  msg_types_and_code_packages_table
  - ADD COLUMN system_id, SMALLINT, +2 bytes
  - DROP INDEX (time,nvps)
  - ADD PRIMARY KEY (system_id, nvps, time)
  - ADD INDEX (time)
  jvm_jvm_metrics_table
  - DROP COLUMN jam, VARCHAR(32), -33 bytes
  - ADD COLUMN jam_id, MEDIUMINT, +3 bytes
  - DROP INDEX (time,jam,pid)
  - ADD PRIMARY KEY (jam_id, pid, time)
  - ADD INDEX (time)
  jvm_jam_jax_rs_metrics_table
  - DROP COLUMN jam, VARCHAR(32), -33 bytes
  - ADD COLUMN jam_id, MEDIUMINT, +3 bytes
  - DROP INDEX (time,jam,pid)
  - ADD PRIMARY KEY (jam_id, pid, time)
  - ADD INDEX (time)
  jvm_jam_jax_rs_op_metrics_table
  - DROP COLUMN jam, VARCHAR(32), -33 bytes
  - ADD COLUMN jam_id, MEDIUMINT, +3 bytes
  - MODIFY COLUMN op_id VARCHAR(128) -> MEDIUMINT, 129 bytes -> 3
  bytes, -125 bytes
  - DROP INDEX (time,jam,pid,op_id)
  - ADD PRIMARY KEY (jam_id, pid, op_id, time)
  - ADD INDEX (time)
  jvm_lock_metrics_table
  - DROP COLUMN jam, VARCHAR(32), -33 bytes
  - ADD COLUMN jam_id, MEDIUMINT, +3 bytes
  - DROP INDEX (time,jam,pid,lock)
  - ADD PRIMARY KEY (jam_id, pid, lock, time)
  - ADD INDEX (time)
  jvm_static_data_table
  - DROP COLUMN jam, VARCHAR(32), -33 bytes
  - ADD COLUMN jam_id, MEDIUMINT, +3 bytes
  - DROP INDEX (jam,pid,start_time)
  - ADD PRIMARY KEY (jam_id, pid, start_time)
  jvm_jmx_app_Kafka_table
  - DROP COLUMN jam, VARCHAR(32), -33 bytes
  - ADD COLUMN jam_id, MEDIUMINT, +3 bytes
  - DROP INDEX (time,jam,pid)
  - ADD PRIMARY KEY (jam_id, pid, time)
  - ADD INDEX (time)
  jvm_jmx_app_JMBeansDrv_table
  - DROP COLUMN jam, VARCHAR(32), -33 bytes
  - ADD COLUMN jam_id, MEDIUMINT, +3 bytes
  - DROP INDEX (time,jam,pid)
  - ADD PRIMARY KEY (jam_id, pid, time)
  - ADD INDEX (time)
  
  
  
  
  BUILD COMMANDS AND INSTRUCTIONS: YES
  #maketpf commands for linux
  maketpf -f CTAL cnvp.o
  maketpf -f CPS0 ccctin.o ccenbk.o ccnucl.o ccstor.o
  maketpf -f CNVI cnvi.o
  maketpf -f CTLG ctlg.o
  maketpf -f CRRN crrn_send_preformatted_data.o
  maketpf CTAL link TPF_VERIFY_LINK_REFS=NO
  maketpf CPS0 link
  maketpf CNVI link TPF_VERIFY_LINK_REFS=NO
  maketpf CTLG link
  maketpf -f CRRS
  maketpf -f CRRI
  maketpf -f CRRT
  maketpf -f CRRW
  maketpf CRRN link
  maketpf CTAL link
  maketpf CNVI link
  
  UPDATED INFORMATION UNITS: YES
  z/TPF and z/TPFDF Migration Guide: PUT 2 and Later
  z/TPF System Performance and Measurement Reference
  
  See your IBM representative if you need additional information.
  
  DOWNLOAD INSTRUCTIONS:
  https://www.ibm.com/support/docview.wss?uid=swg27049604
  
  APAR URL:
  https://transfer.boulder.ibm.com/2022/PJ46608.tar.gz
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  Z/TPF

• Fixed component ID

  5748T1501

Applicable component levels