PJ45657: Provide support for real-time runtime metrics collection.

APAR status

• Closed as new function.

Error description

• See Problem Summary.
  

Local fix

• NA
  

Problem summary

• APAR NUMBER:  PJ45657
  PRODUCT:  z/TPF
  FUNCTIONAL AREA:  NAME VALUE PAIR COLLECTION
  SHIPPED IN YEAR:  2020
  
  ABSTRACT:
  Provide support for real-time runtime metrics collection.
  
  PACKAGE CONTENTS:
  Source Segments:
  (C) base/cntl/ibmconfig.csv
  (C) base/cntl/tpf_app_base.cntl
  (C) base/cntl/tpf_archive_base.cntl
  (C) base/cp/cce4.cpy
  (C) base/cp/cceb.cpy
  (C) base/cp/cceh.cpy
  (C) base/cp/chsz.cpy
  (C) base/cp/cicr.cpy
  (C) base/cp/clhx.cpy
  (C) base/cp/cmis.cpy
  (C) base/cp/ct01.cpy
  (C) base/cp/ct38.cpy
  (C) base/cp/ct41.cpy
  (C) base/cp/cth2.cpy
  (C) base/cp/ctme.cpy
  (N) base/include/tpf/cdcp_collect_lib.h
  (C) base/include/tpf/connapi.h
  (N) base/include/tpf/crrs_send.hpp
  (N) base/include/tpf/crrt.hpp
  (C) base/include/tpf/c_cinfc.h
  (C) base/include/tpf/c_dcdrstruct.h
  (C) base/include/tpf/c_icdc.h
  (C) base/include/tpf/icnvp.h
  (C) base/include/tpf/invpci.hpp
  (C) base/include/tpf/invp_recea30.h
  (C) base/include/tpf/invp_recea31.h
  (N) base/include/tpf/irtmc.hpp
  (C) base/include/tpf/unvp.h
  (C) base/macro/cinfc.mac
  (C) base/macro/icnvp.mac
  (C) base/macro/idatb.mac
  (C) base/macro/invpci.mac
  (C) base/macro/invpi.mac
  (N) base/macro/irtmc.mac
  (C) base/rt/cdcp.c
  (C) base/rt/cdcp.mak
  (N) base/rt/cdcp_add.cpp
  (N) base/rt/cdcp_collect.cpp
  (N) base/rt/cdcp_collect_lib.mak
  (N) base/rt/cdcp_util.cpp
  (C) base/rt/cdcw.c
  (C) base/rt/cnvi.cpp
  (C) base/rt/cnvi.mak
  (C) base/rt/cnvp.cpp
  (N) base/rt/crrc.mak
  (N) base/rt/crri.mak
  (N) base/rt/crrs.mak
  (N) base/rt/crrs_send.cpp
  (N) base/rt/crrt.cpp
  (N) base/rt/crrt.mak
  (N) base/rt/crrt_defs.cpp
  (N) base/rt/crrw.cpp
  (N) base/rt/crrw.mak
  (C) base/rt/cvab.asm
  (N) base/rt/invp_recea31_globals.c
  (N) base/rt/rtmc_globals.cpp
  (N) base/rt/rt_rtmc_display.cpp
  (N) base/rt/rt_rtmc_modify.cpp
  (N) base/rt/rt_rtmc_start.cpp
  (N) base/rt/rt_rtmc_stop.cpp
  (N) base/rt/zrtmc.cpp
  (C) base/tpfrtmc/bin/tpfrtmc.war
  
  Object Only Binaries:
  (C) base/oco/lib/libcpoco.a
  
  BINARIES TO BUILD: YES
  Configuration Independent Binaries:
  (N) base/lib/libcdcp_collect_lib.a
  (N) base/lib/libCNVI.so
  (N) base/lib/libCRRI.so
  (N) base/lib/libCRRS.so
  (C) base/load/CDCO.so
  (C) base/load/CDCP.so
  (C) base/load/CNVI.so
  (N) base/load/CRRC.so
  (N) base/load/CRRI.so
  (N) base/load/CRRS.so
  (N) base/load/CRRT.so
  (N) base/load/CRRW.so
  (C) base/obj/cdcp.o
  (N) base/obj/cdcp_add.o
  (N) base/obj/cdcp_collect.o
  (N) base/obj/cdcp_util.o
  (C) base/obj/cdcw.o
  (C) base/obj/cnvi.o
  (C) base/obj/cnvi2.o
  (N) base/obj/crrs_send.o
  (N) base/obj/crrt.o
  (N) base/obj/crrt_defs.o
  (N) base/obj/crrw.o
  (N) base/obj/invp_recea31_globals.o
  (N) base/obj/rtmc_globals.o
  (N) base/obj/rt_rtmc_display.o
  (N) base/obj/rt_rtmc_modify.o
  (N) base/obj/rt_rtmc_start.o
  (N) base/obj/rt_rtmc_stop.o
  (N) base/obj/zrtmc.o
  
  Configuration Dependent Binaries:
  (C) <sys>/load/CPS0.so
  (C) <sys>/load/CVAB.so
  (C) <sys>/obj/ccctin.o
  (C) <sys>/obj/ccenbk.o
  (C) <sys>/obj/cciisc.o
  (C) <sys>/obj/ccnucl.o
  (C) <sys>/obj/ccstor.o
  (C) <sys>/obj/ccthds.o
  (C) <sys>/obj/cnvp.o
  (C) <sys>/obj/cvab.o
  (C) <sys>/stdlib/libCTAL.so
  (C) <sys>/stdload/CTAL.so
  (C) os390/bin/ppcp.pds
  (C) os390/obj/ccmcdc.o
  (C) os390/obj/stpp.o
  (C) <sys>/load/IPAT.so
  (C) <sys>/load/UCST.so
  (C) <sys>/stdlib/libTPFSTUB.so
  (C) <sys>/stdload/TPFSTUB.so
  (C) <sys>/obj/ipat.o
  (C) <sys>/obj/ucst.o
  
  COMMENTS:
  z/TPF real-time runtime metrics collection provides information
  about the state of the z/TPF system and system resource usage
  by messages and applications. The information is provided in a
  way that monitors, databases, and analysis can easily and
  flexibly consume.
  

Problem conclusion

• SOLUTION:
  Real-time runtime metrics collection provides the following
  advantages:
  Continuous real-time data flow from the z/TPF system without
  using tapes.
  System wide metrics in the form of continuous data collection
  (CDC) data in addition to the highly flexible horizontal and
  vertical name-value pair collection data that represents
  message and application resource usage.
  Easily consumable results from the JSON formatted output from
  Apache Kafka. You can write the results to the database
  solution and format that you choose.
  Robust configuration, for example, a fall back configuration by
  using the high-speed connector.
  The primary data source for real-time analytics pipelines for
  data science analysis, such as statistical analysis, machine
  learning, and other forms of analysis.
  
  The following problems in the base code are addressed by this
  APAR:
  
  The date/time of name-value pair collections was changed from
  reporting local time to UTC time in the MariaDB database for
  the time the collection occurred.
  
  With PJ45892 applied, applications using commit/rollback have
  their name-value pair collection results excluded or shown as 0
  for various counters. The entry CUIU was changed to INCLUDE in
  the ibmconfi.csv file to accomodate application usage of CUIU.
  
  With PJ44321 and PJ45264 applied, an CTL-4 dump can occur from
  applications when running name-value pair collection from
  CPEOWNRC in clhx.cpy because CPNVPC fails to properly restore
  the ECB address space after issuing SWSPC macro calls.
  
  A child ECB created by fork() processing might not be included
  in name-value pair collection data. If the parent ECB didn't
  have any name-value pairs set then the indicator to collect the
  ECB was not copied to the child ECB in cce4.cpy.
  
  ECBs created by Activate on Receipt (AOR) processing should
  always be considered new transactions. If there is already data
  available for the Activate on Receipt then a SWISC CREATE with
  the current system activation number is done. A SWISC CREATE
  always copies the unit of work (UOW) ID, the name-value pair
  area and the collection indicator from the parent to the child
  so the child was not considered a new transaction. This causes
  the name-value pair collection data to be associated with the
  existing transaction instead of a new transaction. Code was put
  into cce4.cpy to check for SWISC CREATE from certain programs
  (like CTSA) that should be considered new transactions. If the
  SWISC CREATE was done from one of these programs then the UOW
  ID, name-value pair area and collection indicator are not
  copied from the parent ECB to the child ECB. chsz.cpy was also
  changed to not reset the collection indicator in the created
  ECB if the sample indicator in the transfer block is zero.
  
  COREQS: NO
  None.
  
  MIGRATION CONSIDERATIONS: YES
  Functional, automation, and operation changes:
  Changed command:
  ZCNVP
  
  New command:
  ZRTMC
  
  New messages:
  CNVP0060E
  CNVP0061E
  RTMC2020E
  RTMC2021E
  RTMC2022E
  RTMC2023E
  RTMC2024E
  RTMC2025I
  RTMC2026E
  RTMC2027E
  RTMC2028E
  RTMC2029E
  RTMC2030E
  RTMC2031E
  RTMC2100I
  RTMC2101I
  RTMC2102E
  RTMC3400E
  RTMC3401E
  RTMC3403E
  RTMC3404E
  RTMC3405E
  RTMC3406E
  RTMC3450E
  RTMC3451E
  RTMC9000E
  RTMC9002E
  RTMC9004E
  RTMC9005E
  RTMC9100I
  RTMC9101E
  RTMC9102E
  RTMC9103E
  RTMC9200I
  RTMC9201E
  RTMC9202E
  RTMC9205E
  RTMC9206E
  RTMC9207E
  RTMC9209E
  RTMC9210E
  RTMC9211E
  RTMC9212E
  RTMC9213E
  RTMC9214E
  RTMC9215E
  RTMC9216E
  RTMC9250I
  RTMC9251E
  RTMC9260I
  RTMC9261E
  RTMC9262E
  RTMC9263E
  RTMC9264E
  RTMC9265E
  RTMC9266E
  RTMC9267E
  RTMC9268E
  RTMC9269E
  RTMC9270E
  RTMC9271E
  RTMC9272E
  RTMC9273E
  RTMC9274E
  RTMC9275E
  RTMC9277E
  RTMC9278E
  RTMC9279E
  RTMC9280E
  RTMC9300I
  RTMC9301E
  RTMC9302E
  RTMC9303E
  RTMC9330I
  RTMC9351E
  RTMC9352E
  
  Application programming interface (API) changes:
  The date/time recorded in name-value pair collection results
  are now recorded as UTC time.
  
  
  
  BUILD COMMANDS AND INSTRUCTIONS: YES
  #maketpf commands for linux
  maketpf -f CTAL cnvp.o
  maketpf -f CPS0 ccctin.o ccenbk.o cciisc.o ccnucl.o ccstor.o
  ccthds.o
  maketpf -f cdcp_collect_lib cdcp_add.o cdcp_collect.o
  cdcp_util.o cdcw.o
  maketpf -f CVAB cvab.o
  maketpf -f CNVI cnvi.o cnvi2.o invp_recea31_globals.o
  maketpf -f CRRS crrs_send.o
  maketpf -f CRRI rtmc_globals.o rt_rtmc_display.o
  rt_rtmc_modify.o rt_rtmc_start.o rt_rtmc_stop.o
  maketpf -f CRRT crrt.o crrt_defs.o
  maketpf -f CRRW crrw.o
  maketpf -f CRRC zrtmc.o
  maketpf -f CDCO cdcw.o
  maketpf -f CDCP cdcp.o
  maketpf -f UCST TPF_VERIFY_LINK_REFS=NO
  maketpf -f TPFSTUB link
  maketpf CTAL link TPF_VERIFY_LINK_REFS=NO
  maketpf CPS0 link
  maketpf -f IPAT
  maketpf cdcp_collect_lib link
  maketpf CVAB link
  maketpf CNVI link
  maketpf CRRS link
  maketpf CRRI link
  maketpf CRRT link
  maketpf CRRW link
  maketpf CRRC link
  maketpf CDCO link TPF_VERIFY_LINK_REFS=NO
  maketpf CDCP link TPF_VERIFY_LINK_REFS=NO
  maketpf UCST link
  maketpf CTAL link
  maketpf CDCO link
  maketpf CDCP link
  #maketpf commands for z/OS
  maketpf -f ppcp ccmcdc.o stpp.o
  maketpf ppcp link
  
  UPDATED INFORMATION UNITS: YES
  z/TPF and z/TPFDF Migration Guide
  z/TPF and z/TPFDF Migration Guide: PUT 2 and Later
  z/TPF Messages (Online, SQLCODEs, and errno Values)
  z/TPF Messages (System Error, Offline, and Program Status Word)
  z/TPF Operations
  z/TPF Program Management
  z/TPF System Performance and Measurement Reference
  
  See your IBM representative if you need additional information.
  
  DOWNLOAD INSTRUCTIONS:
  https://www.ibm.com/support/docview.wss?uid=swg27049604
  
  APAR URL:
  https://transfer.boulder.ibm.com/2020/PJ45657.tar.gz
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  Z/TPF

• Fixed component ID

  5748T1501

Applicable component levels