A fix is available
APAR status
Closed as program error.
Error description
Sniffer trace shows TCP DUP ACK and TCP OUT-OF-ORDER conditions. The duplicate ACK and out-of-order problem is a timing window resulting from running ZIIP IQDIOMULTIWRITE on HiperSockets. Some packets (<32K packets) are being sent to general CPUs while 32K packets are being transmitted across ZIIPs creating a window of opportunity for out-of-order to occur. This condition may occur for both IPv4 and IPv6 flows. This is a base HiperSockets issue related to ZIIP IQDIOMULTIWRITE.
Local fix
Disable ZIIP IQDIOMULTIWRITEwhen using HiperSockets.
Problem summary
**************************************************************** * USERS AFFECTED: * * All users of the IBM Communications Server * * for z/OS Version 2 Release 4 * * Communications Server * * HiperSockets with ZIIP assisted IQDMULTIWRITE * **************************************************************** * PROBLEM DESCRIPTION: * * Duplicate acknowledgements and out-of-order packets reported * * with running HiperSockets or HIperSockets Converged * * Interface (HSCI) with ZIIP IQDIOMULTIWRITE enabled. * **************************************************************** * RECOMMENDATION: * * Apply PTF * **************************************************************** Packets being sent over a HiperSockets interface must contain a total of 32K or more of data for the send operation to be offloaded to zIIP. Intermittently during a streaming socket connection, a TCP send window becomes full where TCP may not be allowed to transmit enough data for the outbound operation to be zIIP offloaded. This means this small data will be scheduled over the general CP to transmit it over Hipersockets. This will cause a race condition between sending over the zIIP and the general CP. We have seen cases where immediately following a send over a general CP, the window opened up and subsequent sends were offloaded to the zIIPs. This condition may cause duplicate acknowledgements (DUP Acks) to occur as the data is arriving out-of-order on the receiver. The DUP acks may cause incidental retransmissions which adds unncessary overhead since the packets are not really lost.
Problem conclusion
Macro updates: tcsndint and t6sndint - have been update to avoid this scenario. Code has been modified to allow a single small outbound operation to be zIIP eligible if we sent previous traffic over the zIIP and only a small amount of time has passed since the previous transmission. This small change will allow the occassional small packet to take the same transmission path as the other sends to keep the data in order. totcpds - has been updated to define new tcb fields to track the time of day of the last zIIP eligible send and if the last send was 32k or larger. Module updates: ezbt6fwr, ezbt6lwr, ezbt6snd, ezbtcfwr, ezbtcsnd and ezbtlfwr - have been recompiled only to pick up the macro changes
Temporary fix
Comments
APAR Information
APAR number
PH11957
Reported component name
TCP/IP MVS
Reported component ID
5655HAL00
Reported release
240
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt / Xsystem
Submitted date
2019-05-09
Closed date
2019-06-11
Last modified date
2019-08-01
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
UI63556
Modules/Macros
EZBT6SND EZBT6LWR EZBTCFWR EZBTLFWR EZBT6FWR EZBTCSND
Fix information
Fixed component name
TCP/IP MVS
Fixed component ID
5655HAL00
Applicable component levels
R240 PSY UI63556
UP19/07/27 P F907
Fix is available
Select the PTF appropriate for your component level. You will be required to sign in. Distribution on physical media is not available in all countries.
[{"Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"SG19M","label":"APARs - z\/OS environment"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"240","Edition":"","Line of Business":{"code":"","label":""}},{"Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"SSCY4DZ","label":"DO NOT USE"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"240","Edition":"","Line of Business":{"code":"","label":""}}]
Document Information
Modified date:
01 August 2019