 | Level: Intermediate Zaid Faydi (zfaydi@au1.ibm.com), Software Engineer,
IBM
Kenichi Yoshimura (kenichi@au1.ibm.com), Software Engineer, IBM
18 Dec 2007 The Fault Analyzer plug-in is the newest addition to the array of
IBM® Rational® Developer for System z™ tools that help you
control every part of the software design and deployment process, including abnormal
endings, known as abends. This article explains how it works and how it differs from
the IBM® Debug Tool for System z™.
Software requirements
Host configuration: The Fault Analyzer feature for the host is included as
part of the IBM® Rational® Developer for System z™ Version 7.1
and later IBM® SMP/E for z/OS™ installation. In addition to the
Rational Developer for System z host configuration, Fault Analyzer for
z/OS V7.1 with PTF UK25564 or later must be configured in your environment.
Consult the Fault Analyzer User's Guide and Reference for further
information (SC19-1253). (See Resources for a link to
the Rational Developer for System z V7.1 Host Installation Guide for full
instructions.)
Client configuration: Fault Analyzer Integration (FAI) is
an optional feature that you can choose to install by using IBM Installation
Manager. This feature is listed in the Fault Analyzer User's Guide and
Reference under "Integration with IBM Application Development and
Problem Determination Tools."
Tip:
The Fault Analyzer Integration feature is also available as a
technology preview for IBM® WebSphere® Developer for System z, Version
7.0 (see Resources for a download link). In addition to
installing and configuring it on the host machine, that version requires
extensions for the host machine that are available in SMP/E format (software management program extended). Also, Fault Analyzer for z/OS V7.1 with PTF UK23264 must
be installed. The client extension can be installed by using the Installation
Manager.
Introduction to this
feature
IBM®
Fault Analyzer for z/OS™
is part of the
z/OS
Problem Determination and Deployment Tools,
which have been around for several years. The Fault Analyzer helps you identify
the cause, analyze the failure, and fix the problem when one of your applications
ends abnormally (an abnormal ending is commonly called an abend). It can
analyze abends of batch, IBM® CICS, IBM® Information Management
System™ (IMS™), IBM® DB2® database, IBM®
WebSphere®, and IBM® WebSphere® MQ™ applications.
The Fault Analyzer supports several different analysis modes:
-
Real-time abend analysis: When a program abends (ends abnormally), the
abend processing is intercepted and the Fault Analyzer is automatically invoked
through an exit appropriate for the processing environment. The Fault Analyzer
analysis records details about the abend in a fault history file (either PDS,
partitioned data set, or PDSE, partitioned data set extended). A fault entry (a
member in a fault history file) contains the result of abending program
analysis.
Note:
The Fault Analyzer feature is invoked to analyze the abending
program through LE CEECXTAN, LE CEEEXTAN, CICS XPCABND, and MVS IEAVTABX exits.
See the Fault Analyzer User's Guide and Reference for further
information (SC19-1253).
-
Real-time SNAP analysis: The Fault Analyzer performs the same analysis
as the real-time abend analysis, except that the SNAP analysis interface is
explicitly invoked by your application where desired.
-
Batch reanalysis: The Fault Analyzer performs the same analysis as the
real-time abend analysis, except that it gets the required information from the
saved information in a fault entry. You can specify additional information to
get a more detailed analysis of the abending program. For example, you can make
a listing (or a side file) available if one was not available during the
real-time analysis. The Fault Analyzer obtains the required information from the
saved minidump (or SYSMDUMP, or both) instead of the abending
program's virtual storage.
-
Interactive reanalysis: The interactive reanalysis presents information
about an abending program as a series of ISPF (Interactive System Productivity Facility) panels. This enables you to choose
the sections of interest. You can also view the contents of storage areas that
are not necessarily formatted in the report generated by other methods. This
provides additional information for certain types of abends (for example,
IBM® CICS® and Java™ technology).
Fault history files are partitioned data set extended (PDSE) data sets that
contain information about faults that have been analyzed by the Fault Analyzer.
Fault entries, which are stored as separate members in the fault history file,
contain the following types of information:
- Real-time analysis key information, such as abend code and failing program
name
- Execution environment details, such as job name, system ID, and the date and
time that the fault occurred
- Associated real-time analysis report (if applicable)
- Saved minidump pages (minidump pages are virtual storage pages that are
accessed during the analysis, which are stored as part of a fault entry)
- Name of associated SYSMDUMP or SVC dump data set (if applicable)
The Fault Analyzer Integration (FAI) feature of Rational Application Developer
for System z, V7.1 and later, enables you to work with history files and fault
entries. This helps you identify the cause of failure in your applications.
How it differs from the
Debug Tool for IBM z/OS
Rational Developer for System z V7.1 helps make the development of your
traditional mainframe applications, Web applications, Web services, and XML-based
interfaces faster and easier. In addition, IBM offers several problem
determination tools (PD tools), including the
Debug
Tool for z/OS,
File
Manager for z/OS,
and
Application
Performance Analyzer for z/OS,
just to name a few. Both Rational Developer for System z V7.1 and PD tools can
improve the efficiency of the development team, as well as the quality of the end
applications.
A development tool such as Rational Developer for System z goes hand in hand with
a debug tool while developing an application. A debug tool is a must-have tool to
step through your application while examining its status, particularly during the
development cycle (as opposed to deployment or production). The Fault Analyzer
feature, by comparison, helps a slightly different user by filling the need as a
problem determination tool in a near-production environment when an unexpected
failure occurs in the application. Based on the status of the abending application
(real-time analysis) or the analysis of system dump captured at the time of
failure, the Fault Analyzer produces a report that describes the cause of failure.
There are three key differences between the Fault Analyzer and the Debug Tool for
z/OS:
- The Fault Analyzer does not require additional resources while your
application is working normally. It is invoked only when an abend occurs.
- When you examine an application, you have a good idea where the problem
exists. The knowledge of problematic areas enables you set break points in the
application and examine the status of the live application. The Fault Analyzer
is invoked when it is needed (when an abend occurs, for instance). You are not
expected to specify the potential problematic areas.
- The Debug Tool enables you to examine the status of your live application as
you go through the code. Fault Analyzer reports the status of your application
at the time of failure, which gives you a snapshot of what was happening in your
application.
Key features of Fault Analyzer Integration for Rational Developer for System z V7.1
The Fault Analyzer plug-in is the newest addition to the suite of Rational
Developer for System z tools that help you control every part of the software
design and deployment process. The plug-in simplifies abend analysis by connecting
to a remote IBM z/OS system and displaying the result of the analysis in the
familiar workspace.
Fault Analyzer Artifacts
view
The Fault Analyzer Artifacts view (Figure 1) enables you to register a set of
history files to browse fault entries to identify the cause of failures in your
applications, and to monitor the contents of history files to alert you when a
failure occurs in one of your applications.
History files are organized in a tree structure in which the root of the tree is
FA Artifacts; which contains children that represent different mainframe
systems. Each system node potentially contains two children, one for containing
all history files for the system and another for containing the view information.
When you double click on a history file element, all fault entries contained in
the selected history file are displayed in the Detailed View (page 5). A view is a
Fault Analyzer concept that enables you to group a set of history files. When you
select a view element, fault entries contained in all history files defined in the
selected view are displayed in the Detailed View.
Note:
A view enables you to group a set of history files, as well
as to define the column layout for the Fault Entry List display in the ISPF
environment. However, the column layout information defined in the view definition
is ignored by the Fault Analyzer Integration feature at this stage.
Views must be defined on the host and retrieved using Retrieve View
Information action from the context menu in the Fault Analyzer Artifacts
view (FA Artifacts). See the "Setting up views" section in
the Fault Analyzer User's Guide and Reference (SC19-1253).
Figure 1. FA Artifacts view
When you invoke the Fault Analyzer
ISPF
interface, a set of history
files that you are using is stored as part of your ISPF profile information. To
retrieve the information of your last-accessed history files:
- In the Artifacts view, select Retrieve Last Access History
Files.
- When theRetrieve ISPF Profile dialog displays, use the arrow for the
drop-down menu to select the following information for these fields (also see
Figure 2):
-
Existing RSE Connection: PTHAPC1
-
ISPF Profile Name: USERID().APC1ISPF.ISPPROF
-
Temporary DSN: USERID().FA.COPY.ISPF.ISPPROF
-
Exec Name: IDIC1.V7R1MO.SIDIEXEC(IDICSXDS)
-
FA Application ID: IDA
Figure 2. Retrieve ISPF Profile
dialog
Table 1 describes these fields.
Table 1. Field names and descriptions
| Field name | Description |
|---|
| Existing RSE Connection | Name of the RSE (remote system explorer) connection profile. The ISPF
profile information is retrieved from the specified host. |
|---|
| ISPF Profile Name | Name of a data set that contains your ISPF profile. |
|---|
| Temporary DSN | This is the name of temporary data set that is used to copy your ISPF
profile. You must have permission to allocate and write to the data set. |
|---|
| Exec Name | Data set name that contains REXX (Restructured Extended Executor, or
exec) to retrieve your ISPF
profile information. The default location is IDI.SIDIEXEC(IDICSXDS). Some
customization is required after the host installation (refer to the exec for
further information). |
|---|
| FA Application ID | Application ID of Fault Analyzer installation on the host. The default value
is IDI (for initial domain identifier). |
|---|
Table 2 is a summary of all actions available in the FA Artifacts view.
Table 2. Summary of FA Artifacts view actions
| Action name | Description |
|---|
| Add new history file | Enables you to add a history file from a particular host to the view. |
|---|
| Retrieve last accessed history file | Enables you to retrieve the set of history files you last used on the host,
based on the information stored in your ISPF profile. |
|---|
| Retrieve view information | Enables you to retrieve information about views, which are stored in a data
set. |
|---|
| Delete from view | Deletes the selected item from the view. |
|---|
| Populate/refresh history file | Refreshes the cached information about the selected history file. |
|---|
| Configure history file monitor | A history file monitor can be configured to monitor the changes made to a
history file on the host. Whenever changes are made to the selected history
file, the user is notified. |
|---|
| Set encoding | Enables you to specify the encoding for the selected system. That is, the
language option used when the fault entry was created. |
|---|
Detailed view
The detailed view (Figure 3) displays the summary of fault entries contained in
the selected history file or view in the FA Artifacts view. This view gives you a
quick summary of what is happening on your system (history file-centric view of
the system). The column headings are configurable, depending on your preference
and area of your interest.
Figure 3. FA details view
From this view, you can invoke the Report browser (see the "Fault
Analyzer Report browser" section). This will enable you to see the
detailed analysis of the selected fault entry or to invoke the Dump browser (see
the section that follows) to browse a hex dump display of minidump pages stored
in the selected fault entry.
Table 3 shows a summary of all actions available from this view.
Table 3. Summary of detailed view actions
| Action name | Description |
|---|
| Browse report | Retrieves the report associated with the selected fault entry and displays
the report. The report is cached locally after it is retrieved. |
|---|
| Browse minidump | Retrieves the minidump pages associated with the selected fault entry and
displays the pages. When the pages associated with a fault entry are
retrieved, they are cached locally. |
|---|
| Delete cached data | Deletes the cached information for the selected fault entry. |
|---|
| Set encoding | Allows you to select an encoding for the system where the selected history
file or view is located. |
|---|
Working with older fault
entries
For a Fault Analyzer Integration client to show fault entries in a history file
correctly, the fault entries must be created with the correct version of the Fault
Analyzer feature, with the appropriate set of options specified. In these cases, a
fault entry cannot be displayed correctly by using the Fault Analyzer Integration
client:
- A fault entry was created by using an older version of the Fault Analyzer.
- A fault entry was created without the WDZClient option.
- A fault entry was created by using the DeferredReport option.
In such cases, the Fault Analyzer Integration client detects the invalidity of
the selected fault entry and displays the Fault Entry Refresh dialog (Figure 4),
so that you can refresh the contents of the fault entry by rerunning the analysis
with the correct set of options.
Figure 4. Fault Entry Refresh
dialog
Fault Analyzer Report
browser
The Fault Analyzer Report browser (Figure 5) is a multi-tabbed browser that
displays the report associated with the selected fault entry. The browser is
typically invoked by running the Browse Report action in the detailed view.
Figure 5. Report browser
Different parts of a report are organized by using tabs for categories that show
along the bottom of the screen. The tabs separate the report into these five
categories and related subcategories:
-
Main Report
-
Fault summary section: A brief description of the fault, which
includes the hyperlinked source line number of the abending program (if
possible) and the abend code. If you click on the source line number, it
displays the source code of the abending program where the failure occurred
(see the "Source code display" section for further
information). If you click on the abend code, it displays the description (see
the "Lookup view" topic that follows).
-
Synopsis section: The synopsis section provides a brief description
of the fault and its analysis.
-
Event Summary
- The Event Summary is a list of all events, in chronological order.
- The Event details subsection provides detailed information about each
event. Additional information associated with the event, such as message
description and the contents of the program's working storage, are
included in this subsection. The source code information or failing machine
instruction is also included here.
-
Abend Job Information
- This section provides information about the abending job associated with the
real-time invocation of the Fault Analyzer feature, such as:
- abend date
- time
- job ID
- job name
- job step name
- execution environment
- Language Environment runtime options
-
Systemwide Information
- This section contains, for example, console messages that are not identified
as belonging to any specific event or CICS system-related information, such as
trace data and 3270 screen buffer contents. Information about open files that
could not be associated with any specific event might also be included here.
- If there is no information in this section, then it does not appear in the
report.
-
Miscellaneous Information
-
Fault Analyzer options: Lists of the Fault Analyzer options that were
in effect at the time of the analysis.
-
Prolog section: Information about the version, release, and
modification level of the Fault Analyzer, as well as the latest APAR
(authorized program analysis report) or PTF (program temporary fix) installed.
-
Epilog section:Provides information about the invocation exists used
and the approximate amount of above-the-line storage allocated during the
analysis, followed by the fault ID assigned. It also includes the time and
date when the report was created.
See the Fault Analyzer User's Guide and Reference (SC19-1253)
for a comprehensive description.
Fault Analyzer Dump
browser
Fault Analyzer Dump browser (Figure 6) is a multi-tabbed browser that enables
you to browse the minidump pages stored in the selected fault entry in a history
file. This browser is typically opened by selecting the Open Mini-Dump
action against a fault entry in the summary view. Minidump pages are
the virtual storage pages that are accessed during analysis and stored as part of
a fault entry.
Figure 6. Minidump Hex Dump
display
Minidump pages are annotated with the key information that the Fault Analyzer
found during the analysis (dictionary data). This dictionary data includes
information such as various control blocks, modules, Multiple Virtual Storage
(MVS) areas, and register values. A summary of the dictionary data associated with
the selected fault entry is presented in the Outline view (Figure 7) or in the
Dictionary Data tab within the Dump browser.
Figure 7. Outline page for Dump
browser
Source code display
The Fault Analyzer supports several different side-file formats to display the
associated source information of an abending program. One of the supported formats
is called the LANGX file, because it is produced by running IDILANGX
utility program against a listing file produced by a compiler.
The Fault Analyzer Integration option supports the source line display of the
abending program if the correct LANGX file was available during the analysis (see
Figure 8). If the correct LANGX file was available during the analysis of your
abending program, the source line number of the program is hyperlinked in the
report. (The location of your LANGX file is specified in the IDILANGX DD card
in your job step.) When you click the source line number (Line 17 in the example
in Figure 8: PROCEDURE DIVISION), it retrieves the LANGX file from the host and
generates the source line for display. The source file opens in your favorite
editor in read-only mode, and the line where the abend occurred is highlighted.
Figure 8. Source line display
The location of LANGX files is specified in the IDILANGX DD statement in your
job. Alternatively, you are given an opportunity to specify the data set names
containing your LANGX files during the fault entry refresh process (Figure 9).
Figure 9. LANGX Data Set Name
dialog
Lookup view
In the Lookup view, you can browse to the description about abend codes,
messages, and other miscellaneous information. Figure 10 shows the description of
IMS user abend U0002. You can browse the information in one of three ways:
- Find the description for a particular abend code, message, or miscellaneous
information by navigating through the tree structure.
- Type in a pattern name in the Search box. All matching abend codes, messages,
and miscellaneous information are displayed in the Results tab in the view. From
the list of matching results, you can browse to find the description.
- An abend code associated with a particular fault entry is hyperlinked in the
report view. Click the hyperlinked abend code to display the description.
Figure 10. Lookup view
You can switch between English or Japanese descriptions for abend codes,
messages, and miscellaneous information.
- Select the Language Setting action from the drop-down menu within the
view.
- When the Lookup Language Setting dialog displays, choose your preferred
language.
Acknowledgements
The authors thank Rod Turner, Clive Nealon, and Oren Nachman for reviewing the
early draft of this article.
Download | Description | Name | Size | Download method |
|---|
| Column heading descriptions | FaultAnalyzerColumnHeadings.pdf | 36KB | HTTP |
|---|
Resources Learn
Get products and technologies
Discuss
About the authors | | | Zaid Faydi is a software engineer in the Australia Development Laboratory. Since joining IBM in 2006, he has worked on IBM Fault Analyzer for z/OS. |
| | | Kenichi Yoshimura is a software engineer in the Australia Development Laboratory. Since joining IBM in 2004, he has worked on several System z software products including IBM Fault Analyzer for z/OS and IBM SCLM Developer Toolkit. |
Rate this page
| |
