QDIO inbound workload queueing

OSA-Express®3 or later features can perform a degree of traffic sorting by placing inbound packets for differing workload types on separate processing queues. This function is called QDIO inbound workload queueing (IWQ). With the inbound traffic stream already sorted by the OSA-Express feature, z/OS® Communications Server provides the following performance optimizations:

  • Finer tuning of read-side interrupt frequency to match the latency demands of the various workloads that are serviced
  • Improved multiprocessor scalability, because the multiple OSA-Express input queues are now efficiently serviced in parallel

When QDIO IWQ is enabled, z/OS Communications Server and the OSA-Express feature establish a primary input queue and one or more ancillary input queues (AIQs), each with a unique read queue identifier (QID) for inbound traffic. z/OS Communications Server and the OSA-Express feature cooperatively use the multiple queues in the following way:

  • The OSA-Express feature directs an inbound packet (received on this interface) that is to be forwarded by the sysplex distributor to the sysplex distributor AIQ. z/OS Communications Server then tailors its processing for the sysplex distributor queue, notably by using the multiprocessor to service sysplex distributor traffic in parallel with traffic on the other queues.
  • The TCP layer automatically detects connections operating in a bulk-data fashion (such as the FTP data connection), and these connections are registered to the receiving OSA-Express feature as bulk-mode connections. The OSA-Express feature then directs an inbound packet (received on this interface) for any registered bulk-mode connection to the TCP bulk-data AIQ. z/OS Communications Server tailors its processing for the bulk queue, notably by improving in-order packet delivery on multiprocessors, which likely results in improvements to CPU consumption and throughput. Like other AIQs, processing for data on the bulk queue can be in parallel with traffic on the other queues.
  • The OSA-Express feature directs an inbound Enterprise Extender packet (received on this interface) to the Enterprise Extender AIQ. This allows z/OS Communications Server to process inbound traffic on the Enterprise Extender queue in parallel with inbound traffic on the other queues for this interface.
  • Start of changeThe OSA-Express feature directs inbound AH packets, ESP packets, and UDP-encapsulated ESP packets that are received on this interface to the IPSec AIQ. This allows z/OS Communications Server to process inbound traffic on the IPSec queue in parallel with inbound traffic on the other queues for this interface.End of change
  • Start of changeThe OSA-Express feature directs inbound packets (received on this interface) that are destined to a zCX server to the zCX AIQ. This allows z/OS Communications Server to process inbound traffic on the zCX queue in parallel with inbound traffic on the other queues for this interface. Additionally, it also allows for this inbound traffic to be processed on a System z Integrated Information Processor (zIIP).End of change
  • If a packet is not directed to an AIQ, the OSA-Express feature directs the packet to the primary input queue.
Requirement: QDIO IWQ is limited to OSA-Express3 or later Ethernet features in QDIO mode that are running on an IBM® System z10 GA3 or later server. For more information, see the 2097DEVICE, 2098DEVICE, or 2817DEVICE Preventive Service Planning (PSP) buckets as appropriate.
Requirement: Start of changeQDIO IWQ for IPSec Start of changeand zCXEnd of change is limited to OSA-Express6S or later Ethernet features in QDIO mode that are running on an IBM z14® or later server. For more information, see the 3906DEVICE or 3907DEVICE Preventive Service Planning (PSP) bucket.End of change
Results: Start of change
  • If you already enabled IWQ on an OSA-Express6S, installed the IWQ for IPSec enablement PTF (TCP/IP APAR PI77649), and have IPSec enabled, z/OS Communications Server will automatically define the IPSec input queue. This new queue will not be backed by CSM storage until the first IPSec tunnel is activated on that stack. Therefore, if you activate any IPSec tunnels, each OSA-Express6S interface on that stack that enables IWQ will consume an additional 36 K of fixed ECSA and an additional amount of fixed 64-bit CSM equal to the READSTORAGE setting for the interface.
  • Start of changeIf you already enabled IWQ on an OSA-Express6S and installed the IWQ for zCX enablement PTFs (VTAM APAR OA58300 and TCP/IP APAR PH16581), z/OS Communications Server will automatically define the zCX input queue. This new queue will not be backed by CSM storage until the first zCX dynamic VIPA is activated on that stack. Therefore, if you activate any zCX dynamic VIPA, each OSA-Express6S interface on that stack that enables IWQ will consume an additional 36 K of fixed ECSA and an additional amount of fixed 64-bit CSM equal to the READSTORAGE setting for the interface.End of change
End of change
Restrictions:
  • QDIO IWQ is not supported for IPAQENET interfaces defined with the DEVICE, LINK, and HOME statements. You must convert your IPAQENET definitions to use the INTERFACE statement to enable this support. For more information, see Steps for converting from IPv4 IPAQENET DEVICE, LINK, and HOME definitions to the IPv4 IPAQENET INTERFACE statement.
  • QDIO IWQ is not supported for a z/OS guest on z/VM® using simulated (virtual) devices, such as virtual switch (VSWITCH) or guest LAN.
  • Bulk-mode TCP connection registration is supported only in configurations in which a single inbound interface is servicing the bulk-mode TCP connection. If a bulk-mode TCP connection detects that it is receiving data over multiple interfaces, QDIO IWQ is disabled for the TCP connection and inbound data from that point forward is delivered to the primary input queue.
  • QDIO IWQ does not apply for traffic that is sent over an OSA port that is shared by the receiving TCP/IP stack when an indirect route (where the next hop and destination IP address are different) is being used; this traffic is placed on the primary input queue. QDIO IWQ does apply when traffic on the shared OSA path uses a direct route (where the next hop and destination IP address are the same).
Guideline: Start of changeThe WORKLOADQ parameter requires an additional amount of fixed 4K CSM HVCOMMON storage per AIQ. The amount of storage consumed per AIQ is based on the amount of storage defined for READSTORAGE for this interface. The bulk AIQ is always backed with this additional CSM storage. The remaining AIQs are not backed by the additional CSM storage until the specific function (EE, SD, or IPSec) is used. The EE AIQ is backed by fixed 4K CSM DSPACE64 storage (instead of HVCOMMON). To verify the amount of CSM storage that is being used for each input queue, display the VTAM® TRLE name that is associated with the interface. The WORKLOADQ parameter also requires an additional 36K of ECSA per AIQ.End of change
Tip: Start of changeThe additional CSM storage consumed by each OSA interface using WORKLOADQ consumes fixed (real) storage. It is recommended that you verify that the additional fixed storage required by enabling WORKLOADQ (per OSA interface) will not approach any of the following limits:
  • The CSM FIXED MAXIMUM value used by Communications Server (use the D NET, CSM command and see the CSM FIXED MAXIMUM value defined in IVTPRM00)
  • The actual real storage available to this z/OS system (see D M=STOR or D M=HIGH)
End of change
Guideline: Start of changeCPU might increase significantly on a busy Sysplex Distributor if QDIO Accelerator is not installed when using IWQ.End of change