New features for IBM Tivoli OMEGAMON XE for DB2 Performance Expert on z/OS, Version 5.1
In October 2010, the latest release of IBM Tivoli OMEGAMON XE for DB2 Performance Expert on z/OS (OMEGAMON XE for DB2 PE), Version 5.1, introduced the Cross-System-Coupling Facility (OMEGAMON/XCF) feature. OMEGAMON/XCF enables the following new functions related to the data sharing group support in OMEGAMON XE for DB2 PE.
- The ability to display CPU percentages from threads that are running on remote LPARs. Because the calculation of CPU percentages requires data from the z/OS system, displaying CPU times on remote threads was not possible without this data. With OMEGAMON/XCF, you can access this z/OS information, and remote CPU times can be displayed.
- The ability to display the Thread Detail Data Set Activity for threads that are running on remote members of a data sharing group. Displaying this activity depends on z/OS information that is available only on the LPAR where the thread is running. With OMEGAMON/XCF you can access this z/OS information, and the Thread Detail Data Set Activity can be displayed.
- The resource names of locks that are being held or waited for on remote members of data sharing groups can be displayed on the Lock Conflict, the Thread Detail Lock Waiter, and the Thread Detail Lock Owner ISPF panels.
OMEGAMON/XCF was added to the data sharing group support in OMEGAMON XE for DB2 PE, Version 4.1. See the following information for more details about features that were also included.
How the OMEGAMON/XCF feature works
When OMEGAMON/XCF is used, multiple OMEGAMON XE for DB2 PE collectors can exchange information by using the z/OS coupling facility.
To enable the OMEGAMON/XCF feature, you must modify options in the high-level-qualifier. RKD2PAR(OMPEOPTS) configuration member, as shown in Listing 1.
Listing 1. OMEGAMON XE for DB2 PE Collector Realtime Customer Options
/*******************************************************************/ /* */ /* OMPEOPTS - OMPE Collector Realtime Customer Options */ /* */ /* Syntax Rules: */ /* 1. Each keyword must be specified in a single record */ /* 2. Blanks in record position 1-72 indicate a comment record */ /* 3. Asterisk(*) in record position 1 indicates a comment record */ /* 4. /* in record position 1-2 indicates a comment record */ /* */ /*******************************************************************/ XCFMODE=ACTIVE Activate XCF option DB2RTCPU=YES Remote CPU option DB2REMIO=YES Remote IIO option |
You must modify the following options:
- XCFMODE
- This option activates OMEGAMON/XCF. Specify ACTIVE.
- DB2RTCPU
- This is the remote CPU time option. Specify YES. When you specify YES, OMEGAMON/XCF gets z/OS CPU usage information by communicating with the OMEGAMON collector-started task on the remote LPAR where the other data sharing group threads are running.
- DB2REMIO
- This is the remote plan IIO option. Specify YES. When you specify YES, OMEGAMON/XCF collects the data set name activity for remote threads.
The Remote Resource Names feature does not require a separate configuration entry in the OMPEOPTS member. If XCFMODE=ACTIVE is specified, the Remote Resource Name feature will be activated.
The following figure shows an example of a system configuration. In this example, the SYSA, SYSB and SYSC LPARs are in the Sysplex. Each LPAR has a OMEGAMON XE for DB2 PE collector and two DB2 subsystems running on it.
Figure 1. Figure 1. System configuration example
A collector-started task runs on each LPAR in the Sysplex. They each join the same XCF group which allows OMEGAMON XE for DB2 PE collectors to exchange data. If data is required for a remote thread, an XCF request is sent to the remote collector. Then, the data is collected and sent back to the collector that is building the data to be displayed.
For more information about enabling OMEGAMON/XCF, see Resources.
Using OMEGAMON/XCF in OMEGAMON XE for DB2 PE, Version 5.1
When you use the remote CPU option, a thread summary is displayed, as shown in the following example.
Listing 2. Thread summary
________________ ZALLT VTM O2 V510./C SZ11 G 04/05/11 20:53:02 2
> Help PF1 Back PF3 Up PF7 Down PF8 Sort PF10 Zoom PF11
> T.A
> Thread Activity: Enter a selection letter on the top line.
> *-All-Idle B-TSO C-CICS D-IMS E-Background F-Dist Allied
> G-Dist DBAC H-Util I-Inact J-Filter K-Functions L-Stored Proc
> M-Triggers N-Sysplex O-Enclaves P-Worksta Q-All+Idle
===============================================================================
> Threads Summary Excluding Idle Threads
THDA
+ *
+ Elapsed Planname CPU Status GetPg Update Commit CORRID/JOBN
+ ---------- -------- ----- ------------ ------ ------ ------ ------------
+ 01-02:16 KO2PLAN 00.0% NOT-IN-DB2 0 0 0
+ 02:10:15.1 DSNESPRR 00.0% SWAPPED-OUT 48 0 0 PAUL
+ 01:59:44.0 ADB 00.1% NOT-IN-DB2 0 0 1 PAUL
+ 01:58:34.8 DSNESPRR 00.2% NOT-IN-DB2 39 0 0 PAUL
+ 01:13:38.7 KO2PLAN 00.0% NOT-IN-DB2 0 0 0 D832DM5S
===============================================================================
|
When you specify DB2RTCPU=YES, you can collect the z/OS data on the LPAR where the thread is running and display the CPU percentages for all the threads, including those on remote LPARs.
When you specify DB2REMIO=YES, the collector retrieves information from the OPME collector that is running on the same LPAR that is also running the thread.
The following example shows the data sets that are being used by this active DB2 thread.
Listing 3. Data sets being used by an active DB2 thread
________________ ZIIO VTM O2 V510./C SZ12 04/05/11 21:07:12 2
> Help PF1 Back PF3
> THREAD INFORMATION: Enter a selection letter on the top line.
> A-THREAD DETAIL B-LOCK COUNTS C-LOCK WAITS D-LOCKS OWNED E-GLOBAL LOCKS
> F-CURRENT SQL G-SQL COUNTS H-DISTRIBUTED I-BUFFER POOL J-GROUP BP
> K-PACKAGES L-RES LIMIT M-PARALLEL TASKS N-UTILITY O-OBJECTS
> P-CANCEL THREAD Q-DB2 CONSOLE *-DSN ACTIVITY S-APPL TRACE T-ENCLAVE
> U-LONG NAMES
===============================================================================
> THREAD DETAIL
PLAN
+ Thread: Plan=DSNESPRR Connid=TSO Corrid=PAUL Authid=PAUL
+ Attach: TSO Userid=PAUL Region asid= 534
+ Package: DSNESM68 Collection=DSNESPRR
+ DB2: SZ11
+ Luwid=DEIBMIPS.IPSAQZ11.C793EDACE009=673
> place a 'T' in front of command to view by tablespace, 'D' by database
iio
+ --Synch I/O-- -Asynch I/O--
+ Dataset Name Delay Count Delay Count
+ -------------------------------------------- ------ ------ ------ ------
+ DBZ1.DSNDBC.DSNDB01.DBD01.I0001.A001 2.5 2 .0 0
+ DBZ1.DSNDBC.DSNDB01.DSNDB01X.I0001.A001 3.2 4 .0 0
+ DBZ1.DSNDBC.DSNDB01.DSNDB1XA.I0001.A001 4.0 4 .0 0
+ DBZ1.DSNDBC.DSNDB01.SYSDBDXA.I0001.A001 2.7 15 6.1 13
+ DBZ1.DSNDBC.DSNDB06.DSNATX02.I0001.A001 3.2 5 .0 0
+ DBZ1.DSNDBC.DSNDB06.DSNDCX05.I0001.A001 6.4 5 .0 0
+ DBZ1.DSNDBC.DSNDB06.DSNDKX03.I0001.A001 15.0 4 .0 0
+ DBZ1.DSNDBC.DSNDB06.DSNDSX01.I0001.A001 105.2 5 .0 0
+ DBZ1.DSNDBC.DSNDB06.DSNDTX01.I0001.A001 1.4 5 .0 0
+ DBZ1.DSNDBC.DSNDB06.DSNDXX07.I0001.A001 7.2 5 .0 0
+ DBZ1.DSNDBC.DSNDB06.SYSTSCOL.I0001.A001 12.6 3 .0 0
+ DBZ1.DSNDBC.DSNDB06.SYSTSIXS.I0001.A001 15.0 2 .0 0
+ DBZ1.DSNDBC.DSNDB06.SYSTSKEY.I0001.A001 26.0 2 .0 0
+ DBZ1.DSNDBC.DSNDB06.SYSTSTAB.I0001.A001 12.6 3 .0 0
+ DBZ1.DSNDBC.DSNDB06.SYSTSTSP.I0001.A001 8.3 3 .0 0
+ DBZ1.DSNDBC.PAULSZ12.PAULSZ12.I0001.A001 43.6 3 65.0 2
===============================================================================
|
Remote resource names and remote lock owners
After the introduction of support for data sharing groups, lock conflicts sometimes occur, and the names of the resources (database name, table space name, and index space name) are not in the local DB2 subsystem's EDM DBD pool. In this case, the lock display panels show resource numbers instead of the resource names. As a result, displayed resource names are introduced by APAR PM35829.
Additionally, on several lock-related panels, detailed information about the remote owner thread are not displayed. As a result, the remote lock thread owner is displayed when you apply APAR PM35122.
The following example shows the LOCKING CONFLICTS panel that was displayed before APAR PM35829.
Listing 4. LOCKING CONFLICTS panel before APAR PM35829
________________ ZLOCKC VTM O2 V420./C SZ11 G 03/28/11 23:50:23 2
> Help PF1 Back PF3 Up PF7 Down PF8 Zoom PF11
> L.
===============================================================================
> LOCKING CONFLICTS
XLOK
+Stat Plan Corrid Type Lvl Resource
+---- -------- ------------ ---- --- ------------------------------------------
+Owned by DB2 Subsystem SZ12PALK Res ID = 0C000006012B000200000001
+OWN DSNESPRR PAUL1 PALK S Res ID = 0C000006012B000200000001
+WAIT DSNESPRR PAUL PALK X Res ID = 0C000006012B000200000001
===============================================================================
|
The following example shows the LOCKS/CLAIMS OWNED BY A THREAD panel before APAR PM35829.
Listing 5. LOCKS/CLAIMS OWNED BY A THREAD panel before APAR PM35829
________________ ZLOCKO VTM O2 V510./C SZ11 04/05/11 21:38:26 2
> Help PF1 Back PF3 Up PF7 Down PF8
> THREAD INFORMATION: Enter a selection letter on the top line.
> A-THREAD DETAIL B-LOCK COUNTS C-LOCK WAITS *-LOCKS OWNED E-GLOBAL LOCKS
> F-CURRENT SQL G-SQL COUNTS H-DISTRIBUTED I-BUFFER POOL J-GROUP BP
> K-PACKAGES L-RES LIMIT M-PARALLEL TASKS N-UTILITY O-OBJECTS
> P-CANCEL THREAD Q-DB2 CONSOLE R-DSN ACTIVITY S-APPL TRACE T-ENCLAVE
> U-LONG NAMES
===============================================================================
> LOCKS/CLAIMS OWNED BY A THREAD
PLAN
+ Thread: Plan=ADB Connid=TSO Corrid=PAUL Authid=PAUL
+ Attach: TSO Userid=PAUL Region asid= 266
+ DB2: SZ12
+ Luwid=DEIBMIPS.IPSARZ12.C793F00657FC=2477
own
+ Lock Ownership Information
+ Percent NUMLKUS = .00 Total Locks Owned = 2
+ Total Catalog Locks = 0 Pageset and Dataset Locks = 0
+ Catalog Pageset Locks = 0 Page/Row Locks = 0
+ Catalog Page/Row Locks= 0 Directory and Other Locks = 0
+ Bind ACQUIRE option = Bind RELEASE option =
+ ISOLATION option = Cursor Stability
+
+ Type Level Resource Number
+ ---- ----- ------------------------------------ ------
+ PSET IS Res id= 0C0000020006000900000000 1
+ SKPT S ColU=NULLID PkU=SYSSH200
+ Token=5359534C564C3031 1
+ ------
+ Total = 2
|
The following example shows the LOCKS/CLAIMS CAUSING A THREAD TO WAIT panel before APAR PM35829.
Listing 6. LOCKS/CLAIMS CAUSING A THREAD TO WAIT panel before APAR PM35829
________________ ZLOCKW VTM O2 V510./C SZ11 04/05/11 21:48:40 2
> Help PF1 Back PF3 Up PF7 Down PF8
> THREAD INFORMATION: Enter a selection letter on the top line.
> A-THREAD DETAIL B-LOCK COUNTS *-LOCK WAITS D-LOCKS OWNED E-GLOBAL LOCKS
> F-CURRENT SQL G-SQL COUNTS H-DISTRIBUTED I-BUFFER POOL J-GROUP BP
> K-PACKAGES L-RES LIMIT M-PARALLEL TASKS N-UTILITY O-OBJECTS
> P-CANCEL THREAD Q-DB2 CONSOLE D-DSN ACTIVITY S-APPL TRACE T-ENCLAVE
> U-LONG NAMES
===============================================================================
> LOCKS/CLAIMS CAUSING A THREAD TO WAIT
PLAN
+ Thread: Plan=DSNTEP10 Connid=BATCH Corrid=PAULLOK2 Authid=PAUL
+ Attach: BATCH JOB Name=PAULLOK2 JOB Asid= 54
+ Package: DSN@EP2L Collection=DSNTEP2
+ DB2: SZ12
+ Luwid=DEIBMIPS.IPSARZ12.C794172334E4=2480
wait
+ Thread Is Currently Suspended Waiting To Obtain The Following Lock
+ Type=TABL Level=IX Resource = 0C0000020006000900000000
+
+ Wait For Lock Time = N/A # to repeat lock timeout = N/A
+ IRLM Time Out Value = 30 Seconds
+
+ The Lock Being Waited On Is Owned By The Following User(s)
+ Plan Connid Corrid Lvl
+ -------- -------- ------------ ---
+ DSNESPRR TSO PAUL X
===============================================================================
|
The following example shows the LOCKING CONFLICTS panel after APAR PM35829.
Listing 7. LOCKING CONFLICTS panel after APAR PM35829
________________ ZLOCKC VTM O2 V420./C SZ11 G 03/28/11 23:50:23 2
> Help PF1 Back PF3 Up PF7 Down PF8 Zoom PF11
> L.
===============================================================================
> LOCKING CONFLICTS
XLOK
+Stat Plan Corrid Type Lvl Resource
+---- -------- ------------ ---- --- ------------------------------------------
+Owned by DB2 Subsystem SZ12PALK DB=PAULSZ12 PS=PAULSZ12
+OWN DSNESPRR PAUL1 PALK S DB=PAULSZ12 PS=PAULSZ12
+WAIT DSNESPRR PAUL PALK X DB=PAULSZ12 PS=PAULSZ12
===============================================================================
|
Now the database name and table space name are displayed for the remote resources as well as the local resources.
The following example shows the LOCKS/CLAIMS OWNED BY A THREAD panel after APAR PM35829. Resource names are displayed.
Listing 8. LOCKS/CLAIMS OWNED BY A THREAD panel after APAR PM35829
________________ ZLOCKO VTM O2 V510./C SZ11 04/05/11 21:38:26 2
> Help PF1 Back PF3 Up PF7 Down PF8
> THREAD INFORMATION: Enter a selection letter on the top line.
> A-THREAD DETAIL B-LOCK COUNTS C-LOCK WAITS *-LOCKS OWNED E-GLOBAL LOCKS
> F-CURRENT SQL G-SQL COUNTS H-DISTRIBUTED I-BUFFER POOL J-GROUP BP
> K-PACKAGES L-RES LIMIT M-PARALLEL TASKS N-UTILITY O-OBJECTS
> P-CANCEL THREAD Q-DB2 CONSOLE R-DSN ACTIVITY S-APPL TRACE T-ENCLAVE
> U-LONG NAMES
===============================================================================
> LOCKS/CLAIMS OWNED BY A THREAD
PLAN
+ Thread: Plan=ADB Connid=TSO Corrid=PAUL Authid=PAUL
+ Attach: TSO Userid=PAUL Region asid= 266
+ DB2: SZ12
+ Luwid=DEIBMIPS.IPSARZ12.C793F00657FC=2477
own
+ Lock Ownership Information
+ Percent NUMLKUS = .00 Total Locks Owned = 2
+ Total Catalog Locks = 0 Pageset and Dataset Locks = 0
+ Catalog Pageset Locks = 0 Page/Row Locks = 0
+ Catalog Page/Row Locks= 0 Directory and Other Locks = 0
+ Bind ACQUIRE option = Bind RELEASE option =
+ ISOLATION option = Cursor Stability
+
+ Type Level Resource Number
+ ---- ----- ------------------------------------ ------
+ PSET IS DB=PAULSZ12 PS=PAULSZ12
+ SKPT S ColU=NULLID PkU=SYSSH200
+ Token=5359534C564C3031 1
+ ------
+ Total = 2
|
The following listing shows the LOCKS/CLAIMS CAUSING A THREAD TO WAIT panel after APAR PM35829. Resource names are displayed.
Listing 9. LOCKS/CLAIMS CAUSING A THREAD TO WAIT panel after APAR PM35829
________________ ZLOCKW VTM O2 V510./C SZ11 04/05/11 21:48:40 2
> Help PF1 Back PF3 Up PF7 Down PF8
> THREAD INFORMATION: Enter a selection letter on the top line.
> A-THREAD DETAIL B-LOCK COUNTS *-LOCK WAITS D-LOCKS OWNED E-GLOBAL LOCKS
> F-CURRENT SQL G-SQL COUNTS H-DISTRIBUTED I-BUFFER POOL J-GROUP BP
> K-PACKAGES L-RES LIMIT M-PARALLEL TASKS N-UTILITY O-OBJECTS
> P-CANCEL THREAD Q-DB2 CONSOLE D-DSN ACTIVITY S-APPL TRACE T-ENCLAVE
> U-LONG NAMES
===============================================================================
> LOCKS/CLAIMS CAUSING A THREAD TO WAIT
PLAN
+ Thread: Plan=DSNTEP10 Connid=BATCH Corrid=PAULLOK2 Authid=PAUL
+ Attach: BATCH JOB Name=PAULLOK2 JOB Asid= 54
+ Package: DSN@EP2L Collection=DSNTEP2
+ DB2: SZ12
+ Luwid=DEIBMIPS.IPSARZ12.C794172334E4=2480
wait
+ Thread Is Currently Suspended Waiting To Obtain The Following Lock
+ Type=TABL Level=IX Resource = DB=PAULSZ12 PS=PAULSZ12
+
+ Wait For Lock Time = N/A # to repeat lock timeout = N/A
+ IRLM Time Out Value = 30 Seconds
+
+ The Lock Being Waited On Is Owned By The Following User(s)
+ Plan Connid Corrid Lvl
+ -------- -------- ------------ ---
+ DSNESPRR TSO PAUL X
===============================================================================
|
Support for data sharing groups in OMEGAMON XE for DB2 PE, Version 4.1
As part of IBM Tivoli OMEGAMON XE for DB2 Performance Expert on z/OS, Version 4.1, data sharing group support for the VTAM Classic Interface was introduced. This support enables all the active threads in a data sharing group to be displayed, even if they are running on a different LPAR in the Sysplex.
On several panels, the field for the DB2 subsystem name is changed to an input field so that switching from one DB2 subsystem to another could be done in context without returning to the main menu and using the Z (RLOG) REDIRECT MONITORING TO ANOTHER DB2 panel.
The Group Mode Indicator input field is added. You can use this field to select single DB2 mode or group mode. Use single mode to monitor one DB2 subsystem or one member of a data sharing group. Use group mode to monitor all the members of a data sharing group. Group mode is not valid for DB2 subsystems that are not members of a data sharing group. If group mode is selected for a DB2 that is not a member of a data sharing group, the KO2O1961W message is displayed, and the mode is reset to single mode.
The following example shows the Threads Summary Excluding Idle Threads panel for one member of a data sharing group. The member name is SZ11, and the group mode indicator is S.
Listing 10. Threads Summary Excluding Idle Threads panel for one member of a data sharing group
________________ ZALLT VTM O2 V510./C SZ11 S 04/05/11 18:47:23 2
> Help PF1 Back PF3 Up PF7 Down PF8 Sort PF10 Zoom PF11
> T.A
> Thread Activity: Enter a selection letter on the top line.
> *-All-Idle B-TSO C-CICS D-IMS E-Background F-Dist Allied
> G-Dist DBAC H-Util I-Inact J-Filter K-Functions L-Stored Proc
> M-Triggers N-Sysplex O-Enclaves P-Worksta Q-All+Idle
===============================================================================
> Threads Summary Excluding Idle Threads
THDA
+ *
+ Elapsed Planname CPU Status GetPg Update Commit CORRID/JOBN
+ ---------- -------- ----- ------------ ------ ------ ------ ------------
+ 00:22:40.5 KO2PLAN 00.0% NOT-IN-DB2 0 0 0 D832DM5S
+ 00:04:24.1 DSNESPRR 00.0% SWAPPED-OUT 48 0 0 PAUL
===============================================================================
|
The following listing shows the Threads Summary Excluding Idle Threads panel for all members of a data sharing group. The group mode indicator is G.
Listing 11. Threads Summary Excluding Idle Threads panel for all members of a data sharing group
________________ ZALLT VTM O2 V510./C SZ11 G 04/05/11 18:57:51 2
> Help PF1 Back PF3 Up PF7 Down PF8 Sort PF10 Zoom PF11
> T.A
> Thread Activity: Enter a selection letter on the top line.
> *-All-Idle B-TSO C-CICS D-IMS E-Background F-Dist Allied
> G-Dist DBAC H-Util I-Inact J-Filter K-Functions L-Stored Proc
> M-Triggers N-Sysplex O-Enclaves P-Worksta Q-All+Idle
===============================================================================
> Threads Summary Excluding Idle Threads
THDA
+ *
+ Elapsed Planname DB2 Status GetPg Update Commit CORRID/JOBN
+ ---------- -------- ----- ------------ ------ ------ ------ ------------
+ 01-00:20 KO2PLAN SZ12 NOT-IN-DB2 0 0 0
+ 00:14:52.7 DSNESPRR SZ11 SWAPPED-OUT 48 0 0 PAUL
+ 00:04:23.2 ADB SZ12 NOT-IN-DB2 0 0 1 PAUL
+ 00:03:14.0 DSNESPRR SZ12 NOT-IN-DB2 39 0 0 PAUL
+ 00:01:17.2 KO2PLAN SZ11 NOT-IN-DB2 0 0 0 D832DM5S
===============================================================================
|
Now all of the active threads for the entire data sharing group are displayed. OMEGAMON XE for DB2 PE can monitor all the members of a data sharing group by using the local monitored member and requesting the DB2 instrumentation facility interface (IFI) from one or more remote members. This method has some limitations. Some of the fields displayed are from sources other than the DB2 IFI. To calculate the percentage of CPU time that is used by a single thread requires access to z/OS information, so it cannot be computed for threads that are running on remote LPARs by using only information from the instrumentation facility component identifier (IFCID) records. In V4.1, on the group thread display we substitute the DB2 subsystem member name in the data sharing group for the CPU time because it cannot be displayed. However, in V5.1, CPU percentages can be shown for all threads in the data sharing group.
OMEGAMON XE for DB2 PE continues to expand the support for DB2 data sharing groups by using the new OMEGAMON/XCF feature. OMEGAMON/XCF is a feature for general purposes, and it will be used to expand the data sharing group functionality of OMEGAMON XE for DB2 PE in later releases.
Learn
- For more information about enabling OMEGAMON/XCF, see
IBM Tivoli OMEGAMON XE for DB2 Performance Monitor on z/OS Version 5.1.0 Configuration and Customization
topic in the Information Center.
- Learn how Tivoli
OMEGAMON XE for DB2 Performance Expert on z/OS evaluates the efficiency of and
optimizes performance from your DB2 in a z/OS environment.
- Explore how the IBM performance
management solution can improve productivity and profitability in your business.
- In the DB2 for z/OS area on developerWorks,
get the resources you need to advance your DB2 skills.
- Learn more about Information Management at the developerWorks Information Management
zone. Find technical documentation,
how-to articles, education, downloads, product information, and
more.
- Stay current with
developerWorks technical events and webcasts.
- Follow developerWorks on
Twitter.
Get products and technologies
- Build your next
development project with
IBM trial software,
available for download directly from developerWorks.
Discuss
- Participate in the discussion forum.
- Check out the
developerWorks
blogs and get involved in the
developerWorks community.
Paul Kenney is a Senior Software Engineer and has been doing software development for many years on DB2-related tools. He is an Architect and a Development Lead for the IBM Tivoli OMEGAMON XE for DB2 Performance Expert on z/OS. He is also an IBM Certified Solutions Expert for DB2 Database Administration.




