IBM Support

Troubleshooting and Diagnosis

White Papers


Abstract

This white paper focuses on best practices for troubleshooting and diagnosis of general problems encountered with the IBM Data Virtualization Manager for z/OS technology. For specific technical problems or inability to procure PTFs or updated client components, open a problem ticket with IBM Support.

Content

When you encounter problems with your environment or application, it is always good to know where to start and how to characterize the situation. This chapter will focus on initial assessment, must gather, and Server Trace practices in displaying, viewing, and gathering details. In the case where IBM Support is required, some direction is also provided specifically to “must gather” materials.
Initially characterizing problems
Proper context around the environment, configuration, accessing applications and active workloads are critical to determining the severity of a problem and tracking down its resolution. The following questions are great to ask and use to assess the problem and overall context.
  1. Is this a new or existing application?
  2. Has anything changed in the environment, application, or configuration? Y/N
  3. If failure occurred, what is the display or recorded message?
  4. Did the problem result in a down or unproductive system? Y/N
  5. Did the problem occur in a Test, Development, QA, or Production environment?
  6. Is the problem encountered delaying a production deployment? Y/N
  7. What is the time frame for a resolution to your problem?
  8. Can you reproduce the problem and capture output to support your experience? Y/N
  9. If your problem can be reproduced, provide the exact steps for reproduction
  10. Provide sample SQL or Source Code associated that impacts the current problem

Must Gather information
If you are unsuccessful in searching available Techdocs for your problem, you can gather details about your environment, application and behavior experienced and initiate a problem ticket with IBM Support.
DVM for z/OS version
IBM distributes and supports one version of IBM Data Virtualization Manager for z/OS. IBM actively supports version (1.1.0).
High Module date
Provide a complete copy of your DVM server JOBLOG. This file contains the following information and more, which helps the IBM support team. The job log is a record of job-related information for the programmer.
For example:
AVZ0340I SVFXLevel = 2016/02/18 13:40:50 SVFX0000 01.01.00
AVZ0340I High module OPTPIN assembled at: 2016/02/18 13.40 
PTF Level
The PTF level can be found that uses this sample job:

//GIMSMP   EXEC PGM=GIMSMP,REGION=0M              
//SMPCSI   DD  DISP=SHR,DSN=<HLQSMP>.GLOBAL.CSI   
//SMPLOG   DD  DUMMY                              
//SMPOUT   DD  SYSOUT=*                           
//SMPCNTL  DD  *                                  
    SET BDY(<TGTZONE>) .                          
    LIST  SYSMODS  XREF .                         
/*                                                 
Operating System
  • Provide the level of z/OS in use
  • Provide the IBM Z z/architecture currently in use (for example, z13, z14, z15)
DVM server environment
  • Provide the Client operation system where the DVM Studio is installed
  • Provide both the version of MS Windows and whether the workstation
  • is 32-bit or 64-bit
  • Provide the build version of the DVM Studio
  • Provide the current version of JDBC or ODBC driver in use
Capture Trace Browse from the DVM server ISPF panel
Trace Browse is a facility in which the DVM Server logs critical events that can be viewed to help diagnose, debug, and correct problems encountered with the configuration or execution of the solution.  The trace adds records to a trace buffer maintained in virtual storage. When a session completes, the trace records are automatically saved in a VSAM data set. 
Refer to chapter 6 in this Redbook for more detailed information on Server Traces. An appendix item also exists in this Redbook on this topic, as well as interspersed into other chapters, specific to data source connections and interface methods.
Capture a copy of the Trace Browse as follows:
DVM for z/OS has one principle mechanism for diagnostics and troubleshooting. During the initial configuration of the DVM Server, the system administrator is prompted to designate a data set for traces. The traces can be viewed from the DVM Server ISPF panel or the desktop DVM Studio.
ISPF Panel

Figure 1 - Selecting Server Trace from the DVM Server ISPF panel
Server Trace inline tutorial
Figure 2 - Server Trace inline tutorial

Paging through the tutorial displays DISPLAY, LOCATE, FIND, RFIND and PROFILE. Double-clicking any of these words will ZOOM into details for the command. For example, selecting DISPLAY presents information about that command, as shown in Figure 3.
ZOOM on DISPLAY
Figure 3 - ZOOM on Display

Display and view server Traces
Server Trace information can be viewed and printed on both the ISPF interface, as well as the DVM Studio.
Using the ISPF panel to display and view server Trace
Typing ‘1’ at the Command prompt in the Server Trace ISPF panel displays the latest Server Trace written, as shown in Figure 4.

  1. Connect to the DVM ISPF panel
  2. Select B (Server Trace)
  3. On the command line, issue a D MSGNO T CNID and press ENTER
  4. Print a block of the trace, type SS over the MSGNO column from the lines to copy from and SS over the MSGNO column for where to copy to, and press ENTER. The printed block of the trace will write the trace to the ISPF list data set.
  5. Issue the LIST command from the command line and specify option 3 to keep the listed data set and allocate a new one
  6. Terse the trace data set and attach it to your problem Ticket
Server Trace display
Figure 4 - Server trace display
The display shows the timestamp for the trace record without a specific date the trace record was written. Typing ‘D DATE TIME’ at the Command line prompt displays both date and time in DDMMM HH:MM:SS format, as shown in Figure 5.
Server Trace display with date and time
Figure 5 - Server Trace display with date and time
This panel in the Server Trace Facility shows summary steps that are being traced. Double-clicking any line in this view will ZOOM into the capture for the particular record. This example in Figure 6 is the processing of an SQL statement with associated detail information.
Server Trace display with detail for SQL statement
Figure 6 - Server Trace display with detail for SQL statement
This type of Server Trace information is helpful when working with IBM Support. At times, the system administrator might need to capture a Server Trace for a particular situation that either happened or that can be re-created in a separate environment send for further analysis.
Capture and print server Trace output
In order to capture and print the Server Trace output, the administrator needs to export the Server Trace to a data set using the following steps.
  1. From the Server Trace Command prompt type 'D MSGNO T TCB' and press <Enter>
  2. Type 'PP' over the displayed MSGNO column for the start line of the block to copy then 'PP' over the MSGNO column for the last line of the block to copy.
    Tip: Capture additional lines before and after the problem error as a good practice
    (Substitute "SS" for "PP" for "Zoomed" Trace, to get all of the underlying content you would get by double-clicking)
  3. Issue the LIST command from the SErver Trace Command prompt
  4. Specify option'3' to keep the list data set and allocate a new one
  5. Download the list data set and attach the file to your IBM Support problem ticket

Using the DVM Studio to display and view server Trace
Chapter 6 in this Redbook fully describes how to use the DVM Studio to display, view, capture, and export Server Trace detail.

Using the DVM Studio to diagnose SQL results
A more accessible form of debugging is in the SQL Results tab and the Console View.  This example introduces a stray keystroke in the SQL that makes the Virtual table name invalid, as shown in the SELECT statement in Figure 7, resulting in the SQL error attempting to execute the statement.

SQL results error message from the SQL tab

Figure 7 - SQL results error message from the SQL tab

The SQL Results show the error “Unable to process map ASTAFFVS`”.  Also, of interest to tech support and developers is the version number of the JDBC driver  [DV][JDBC Driver][3.1.201912091012].  
The Console gives more detailed messaging, as shown in Figure 8.

Detail message from the Console

Figure 8 - Detailed message from the Console
In the event, the Console View is not open, it is possible to use the Show View drop-down menu and select “Other”.  Typing the first few letters of “Console” displays the icon as a selection for the Console that can be selected to continue diagnosis, as shown in Figure 9.
Show View input field
Figure 9 - Show View input field
Consolidate server Trace over multiple DVM servers
This chapter focuses primarily on a single DVM Server subsystem for working with Server Trace. However, when multiple DVM Servers are participating in a single LPAR, across LPARs, or part of a Sysplex, there is a way to adjust the settings in the IN00. This helps the administrator centralize all DVM subsystem Server Traces to a single repository so that analysis and troubleshooting can be performed more easily.
Edit the INOO file to enabled the DVM Instrumentation Server (SIS)
Figure 10 - Edit the INOO file to enable the DVM Instrumentation Server (SIS)
Enter the fields as described and change the text “DontDoThis” to “DoThis”, as shown in Figure 11.
Add code to define the SISXCF server
Figure 11 -  Add code to define the SISXCF server
Figure 12 demonstrates an example of combining the logs from two different LPARS. The fields inside the “if” statements are used to identify which records come from each LPAR.
Combined Server Trace logs from two (2) DVM Servers over two (2) LPARs
Figure 12 - Combined server Trace logs from two DVM Servers over two LPARs
Figure 13 displays combined Server Trace output for both DVM Servers. RS28AVZ1BERT is one LPAR, RS22AVZ1ERNIE is a second LPAR and SISLOCALRS28 is the local DVM Server for the present environment.
Server Trace output for both DVM Servers
Figure 13 - Server Trace output for both DVM Servers
Search Techdocs for answers
Answers to many problems can be found on the IBM Support website located at https://www.ibm.com/support/home/.Typing keywords into the search field is always the first recommendation when you encounter a problem in your running environment. There are 150+ Techdocs on IBM Data Virtualization Manager for z/OS that can assist in solving your technical issue before opening an IBM Support problem ticket as shown in Figure 14.
IBM Support website
Figure 14 - IBM Support website
Table 1 lists frequently encountered problems that users encounter when first starting to work with DVM for z/OS. This shortlist represents a shortlist of topics that are available by searching the web. 
Populat Technotes
Category
Data
SSL
JDBC Gateway server
Performance
Load Balance Configuration
DVM Studio
Server Trace
z/OS Connect Service Provider Logging
Service ISPF activity display
Others
Table 1 - Popular DVM for z/OS Technotes
Additionally, DVM for z/OS also has a new IBM Community that is available to submit questions, download sample videos, and so on. These Videos used to be part of a Youtube.com channel that has been removed. IBM also produces IBM Demos that can also help introduce new users to the product through Demos, Product Tours, and
Hands-On Labs.

[{"Type":"SW","Line of Business":{"code":"LOB10","label":"Data and AI"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SS4NKG","label":"IBM Data Virtualization Manager for z\/OS"},"ARM Category":[{"code":"a8m0z000000cxAOAAY","label":"SQL Engine"}],"Platform":[{"code":"PF035","label":"z\/OS"}],"Version":"All Version(s)"}]

Product Synonym

DVM

Document Information

Modified date:
01 July 2021

UID

ibm16447451

Page Feedback