A fix is available
APAR status
Closed as program error.
Error description
The system is IPL'd and the following message starts coming out. . DFHAP0002 A severe error (code X'2526') has occurred in module DFHERM. . It appears that Task 01484 "@004" has been abending over and over getting AITC abends. . 05D4 10:48:26 0278 TRAN_ABEND_PERCOLATE ---/AITC DFHPCP 05D5 10:48:26 0278 TRAN_ABEND_PERCOLATE ---/AITC DFHMIR 05D6 10:48:26 0278 TRAN_ABEND_PERCOLATE ---/AITC DFHAPPG 05D7 10:48:30 0278 TRAN_ABEND_PERCOLATE ---/AITC DFHPCP 05D8 10:48:30 0278 TRAN_ABEND_PERCOLATE ---/AITC DFHMIR 05D9 10:48:30 0278 TRAN_ABEND_PERCOLATE ---/AITC DFHAPPG 05DA 10:49:51 0278 TRAN_ABEND_PERCOLATE ---/AITC DFHPCP 05DB 10:49:51 0278 TRAN_ABEND_PERCOLATE ---/AITC DFHMIR 05DC 10:49:51 0278 TRAN_ABEND_PERCOLATE ---/AITC DFHAPPG 05DD 10:49:53 0278 TRAN_ABEND_PERCOLATE ---/AITC DFHPCP 05DE 10:49:53 0278 TRAN_ABEND_PERCOLATE ---/AITC DFHMIR 05DF 10:49:53 0278 TRAN_ABEND_PERCOLATE ---/AITC DFHAPPG . The following Exception Trace Entry is in the Internal Trace. . 2526 ERM *EXC* - RMLN_SET_LINK_FAILED FUNCTION(SET_LINK) RESPONSE(EXCEPTION) REASON(LINK_UNKNOWN) LINK_ID_BUFFER(261D58CC , 00000008 , 00000008) . TASK-01484 KE_NUM-0278 TCB-L8009/00A9F088 RET-A78C89BC TIME-10:51:02.2987290473 INTERVAL-00.0040752456 =012361= . This Task has been in DB2, there is a TIE Control Block for it in the RM Domain. At offset x'64' there should be a LINK TOKEN which in this case is zeros. . TIE.DSNCSQL 261D5870 Task Interface Element . 0000 015E6EE3 C9C56060 C4E2D5C3 E2D8D340 0020 20800000 FF515100 00000000 00084B9C 0040 E34BC9C9 C2E2E8E2 D2F43E0B 0CA51B00 0060 40404040 00000000 2588E260 80800000 0080 00DE6EC4 C6C8C4F2 D3D6E340 40404040 00A0 261D5630 0011B020 261D5630 2592267C 00C0 00000000 00000000 00000000 00000000 00E0 A0000000 00000000 00000000 00000000 0100 D7C2C2C4 F9F0F040 40404040 40404040 0120 C9C9C2E2 E8E2D240 F43E0B0C A51BC6D9 0140 00000000 00000018 0D050001 00000000 . In UEH=3, all the TIEs have something there, except the TIE for Task 01484 which has all zeroes there. . It seems DFHMIR+x'1be6' (275F95E6) is R14 for the trace entries that look like the following. A link token of zero is not going to work for RMLN SET_LINK: . RM 0301 RMLN ENTRY - FUNCTION(SET_LINK) LINK_TOKEN(00000000) RECOVERY_STATUS(UNNECESSARY) . which all are failing with . RM 0302 RMLN EXIT - FUNCTION(SET_LINK) RESPONSE(EXCEPTION) REASON(LINK_UNKNOWN) TASK-27303 KE_NUM-0231 TCB-QR /00AFC128 RET-A75F95E6 TIME-16:25:25.2351712976 INTERVAL-00.0000080000 . Link Link Owning || |||| || ||| Client Client Remote Link Address Token UOW Addr || |||| || ||| Name Token Access Id Id . 2A2AAAE0 FFF80000 260096D0 NN NNNM UY NNN SOCK 0102FEB6 DEF 2A2AAC10 FFF90000 260096D0 NN NNNM UY NNN SOCK 0102FEB7 DEF 2A2AAD40 FFFA0000 260096D0 NN NNNM UY NNN SOCK 0102FEB8 DEF 2A2AAE70 FFFB0000 260096D0 NN NNNM UY NNN SOCK 0102FEB9 DEF 2A2AB030 FFFC0000 260096D0 NN NNNM UY NNN SOCK 0102FEBA DEF 2A2AB160 FFFD0000 260096D0 NN NNNM UY NNN SOCK 0102FEBB DEF 2A2AB290 FFFE0000 260096D0 NN NNNM UY NNN SOCK 0102FEBC DEF 2A2AB3C0 FFFF0000 260096D0 NN NNNM UY NNN SOCK 0102FEBD DEF 2A2AB4F0 00000000 260096D0 NN NNNM UY NNN SOCK 0102FEBE DEF 2A2AB620 00000000 260096D0 NN NNNM UY NNN SOCK 0102FEBF DEF 2A2AB750 00000000 260096D0 NN NNNM UY NNN SOCK 0102FEC0 DEF 2A2AB880 00000000 260096D0 NN NNNM UY NNN SOCK 0102FEC1 DEF 2A2AB9B0 00000000 260096D0 NN NNNM UY NNN SOCK 0102FEC2 DEF 2A2ABAE0 00000000 260096D0 NN NNNM UY NNN SOCK 0102FEC3 DEF 2A2ABC10 00000000 260096D0 NN NNNM UY NNN SOCK 0102FEC4 DEF 2A2ABD40 00000000 260096D0 NN NNNM UY NNN SOCK 0102FEC5 DEF 2A2ABE70 00000000 260096D0 NN NNNM UY NNN SOCK 0102FEC6 DEF 2A2AC030 00000000 260096D0 NN NNNM UY NNN SOCK 0102FEC7 DEF 2A2AC160 00000000 260096D0 NN NNNM UY NNN SOCK 0102FEC8 DEF 2A2AC290 00000000 260096D0 NN NNNM UY NNN SOCK 0102FEC9 DEF 2A2AC3C0 00000000 260096D0 NN NNNM UY NNN SOCK 0102FECA DEF 2A2AC4F0 00000000 260096D0 NN NNNM UY NNN SOCK 0102FECB DEF 2A2AC620 00000000 260096D0 NN NNNM UY NNN SOCK 0102FECC DEF . It looks like RM Domain lost it when the link token overflowed FFFF0000 to 00000000 . . This is very similar to PM02946. In this case every call to Datapower is failing. From looking at residual receive buffers it appears the caller is not authorised. Datapower closes the connection after each failure. The application program just repeatedly issues WEB CONVERSE commands. CICS helpfully reconnects to the server each time. To do that DFHWBCL first issues a DFHSOCK close to close the previous socket. DFHWBCL is not also issuing a DFHSOCK RELINQUISH call so the RM link remains allocated. Over time these build up and exceed the x'FFFF' limit. . If the application corrects the authorization error then CICS would simply reuse the existing connection and there wouldn't be any issue. . Additional Symptom(s) Search Keyword(s): KIXREVACC
Local fix
If the application corrects the authorization error then CICS would simply reuse the existing connection and there wouldn't be any issue.
Problem summary
**************************************************************** * USERS AFFECTED: All CICS Users. * **************************************************************** * PROBLEM DESCRIPTION: DFHAP0002 Severe Error code x'2526' has * * occurred in DFHERM * **************************************************************** A CICS Web client application issues repeated WEB CONVERSE commands without checking a response. For each converse, DFHWBCL will check the state of the connection and in this case it is closed. DFHWBCL will close the socket and establish a new connection. When DFHWBCL closes the socket it fails to delete and free the associated RM LINK for the socket, this eventually exhausts all available link tokens. Keywords: ERM_SET_LINK_FAIL
Problem conclusion
DFHWBCL has been changed to issue a SOCK RELINQUISH call before it closes the socket.
Temporary fix
Comments
APAR Information
APAR number
PI98172
Reported component name
CICS TS Z/OS V5
Reported component ID
5655Y0400
Reported release
800
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt / Xsystem
Submitted date
2018-05-21
Closed date
2018-05-23
Last modified date
2018-06-04
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
UI56016 UI56017 UI56019 UI56020
Modules/Macros
DFHWBCL
Fix information
Fixed component name
CICS TS Z/OS V5
Fixed component ID
5655Y0400
Applicable component levels
R000 PSY UI56019
UP18/05/24 P F805
R100 PSY UI56017
UP18/05/24 P F805
R800 PSY UI56020
UP18/05/24 P F805
R900 PSY UI56016
UP18/05/25 P F805
Fix is available
Select the PTF appropriate for your component level. You will be required to sign in. Distribution on physical media is not available in all countries.
[{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SSGMGV","label":"CICS Transaction Server"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"5.1","Edition":"","Line of Business":{"code":"LOB35","label":"Mainframe SW"}},{"Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"SG19M","label":"APARs - z\/OS environment"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"5.1","Edition":"","Line of Business":{"code":"","label":""}}]
Document Information
Modified date:
04 June 2018