IBM Support

IT33662: ENABLE.AUTO.COMMIT AND ASYNC OFFSET COMMIT OPTIONS FOR KAFKA CONSUMER NODE.

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as requirement.

Error description

  • This APAR introduces following features for KafkaConsumer node.
    
    1) If the "enable.auto.commit" property is set to "true" in the
    Kafka properties file, this setting is now honoured and takes
    precedence over the "Commit message offset in kafka" setting on
    the KafkaConsumer node.
    
    2) The "Commit message offset in kafka" property on the
    KafkaConsumer node updates the position of the kafka consumer
    saved in the kafka Server as each message is processed by the
    message flow. The KafkaConsumer node waits for the save of
    consumer position to be completed before the message is
    processed by the message flow ensuring an at-most-once level of
    message reliability. The action of waiting for the response to
    the saving of the consumer offset introduces a significant delay
    before the message is processed by the message flow and in turn
    the maximum throughput of the message flow. This APAR optionally
    allows the saving of the consumer offset to be performed
    asynchronously, which means that the request to save the
    consumer offset is still made but we do not wait for the
    response before the message is processed by the message flow
    (this style of operation is called asynchronously committing the
    offset). Setting the environment variable
    "MQSI_KAFKA_USE_SYNC_OFFSET_COMMIT=FALSE" in the environment of
    the integration server enables this asynchronous mode of
    operation.
    

Local fix

  • N/A
    

Problem summary

  • ****************************************************************
    USERS AFFECTED:
    All users of IBM AppConnect Enterprise V11.0 and IBM Integration
    Bus V10.0 using the KafkaConsumer node.
    
    
    Platforms affected:
    z/OS, MultiPlatform
    
    ****************************************************************
    PROBLEM DESCRIPTION:
    This APAR introduces following features for KafkaConsumer node.
    
    1) If the "enable.auto.commit" property is set to "true" in the
    Kafka properties file, this setting is now honoured and takes
    precedence over the "Commit message offset in kafka" setting on
    the KafkaConsumer node.
    
    2) The "Commit message offset in kafka" property on the
    KafkaConsumer node updates the position of the kafka consumer
    saved in the kafka Server as each message is processed by the
    message flow. The KafkaConsumer node waits for the save of
    consumer position to be completed before the message is
    processed by the message flow ensuring an at-most-once level of
    message reliability. The action of waiting for the response to
    the saving of the consumer offset introduces a significant delay
    before the message is processed by the message flow and in turn
    the maximum throughput of the message flow. This APAR optionally
    allows the saving of the consumer offset to be performed
    asynchronously, which means that the request to save the
    consumer offset is still made but we do not wait for the
    response before the message is processed by the message flow
    (this style of operation is called asynchronously committing the
    offset). IBM AppConnect Enterprise V11.0 introduced a new node
    property 'Wait for message offset commit to complete' in the
    KafkaConsumer node to control asynchronous commit of message
    offsets. Setting the environment variable
    "MQSI_KAFKA_USE_SYNC_OFFSET_COMMIT=FALSE" in the environment of
    the integration server enables this asynchronous mode of
    operation in IBM Integration Bus V10.0.
    

Problem conclusion

  • The product now honors "enable.auto.commit" property set in the
    Kafka properties file and also provides option to enable
    asynchronous commit of message offsets.
    

Temporary fix

Comments

  • New properties introduced for KafaConsumer node
    

APAR Information

  • APAR number

    IT33662

  • Reported component name

    INTEGRATION BUS

  • Reported component ID

    5724J0540

  • Reported release

    A00

  • Status

    CLOSED REQ

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2020-07-23

  • Closed date

    2020-09-01

  • Last modified date

    2020-09-01

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

Applicable component levels

[{"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Product":{"code":"SSNQK6","label":"IBM Integration Bus"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"10.0","Line of Business":{"code":"LOB36","label":"IBM Automation"}}]

Document Information

Modified date:
02 September 2020