IBM Support

PI99138: Repeated delivery of Batch job dispatch JMS message resulting in ClassCastException each time.

Fixes are available

18.0.0.2: WebSphere Application Server Liberty 18.0.0.2
18.0.0.3: WebSphere Application Server Liberty 18.0.0.3
18.0.0.4: WebSphere Application Server Liberty 18.0.0.4
19.0.0.1: WebSphere Application Server Liberty 19.0.0.1
19.0.0.2: WebSphere Application Server Liberty 19.0.0.2
19.0.0.3: WebSphere Application Server Liberty 19.0.0.3
19.0.0.4: WebSphere Application Server Liberty 19.0.0.4
19.0.0.5: WebSphere Application Server Liberty 19.0.0.5
19.0.0.6: WebSphere Application Server Liberty 19.0.0.6
19.0.0.7: WebSphere Application Server Liberty 19.0.0.7
19.0.0.8: WebSphere Application Server Liberty 19.0.0.8
19.0.0.9: WebSphere Application Server Liberty 19.0.0.9
19.0.0.10: WebSphere Application Server Liberty 19.0.0.10
19.0.0.11: WebSphere Application Server Liberty 19.0.0.11
19.0.0.12: WebSphere Application Server Liberty 19.0.0.12
20.0.0.1: WebSphere Application Server Liberty 20.0.0.1
20.0.0.2: WebSphere Application Server Liberty 20.0.0.2
20.0.0.3: WebSphere Application Server Liberty 20.0.0.3
20.0.0.4: WebSphere Application Server Liberty 20.0.0.4
20.0.0.5: WebSphere Application Server Liberty 20.0.0.5
20.0.0.6: WebSphere Application Server Liberty 20.0.0.6
20.0.0.7: WebSphere Application Server Liberty 20.0.0.7
20.0.0.8: WebSphere Application Server Liberty 20.0.0.8
20.0.0.9: WebSphere Application Server Liberty 20.0.0.9
20.0.0.10: WebSphere Application Server Liberty 20.0.0.10
20.0.0.11: WebSphere Application Server Liberty 20.0.0.11
20.0.0.12: WebSphere Application Server Liberty 20.0.0.12
21.0.0.3: WebSphere Application Server Liberty 21.0.0.3
21.0.0.4: WebSphere Application Server Liberty 21.0.0.4
21.0.0.5: WebSphere Application Server Liberty 21.0.0.5
21.0.0.6: WebSphere Application Server Liberty 21.0.0.6
21.0.0.7: WebSphere Application Server Liberty 21.0.0.7
21.0.0.8: WebSphere Application Server Liberty 21.0.0.8
21.0.0.9: WebSphere Application Server Liberty 21.0.0.9
21.0.0.1: WebSphere Application Server Liberty 21.0.0.1
21.0.0.2: WebSphere Application Server Liberty 21.0.0.2

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as program error.

Error description

  • The problem will occur when using the Liberty Batch JMS
    dispatcher/executor function with an executor configured to
    use the "group security"  function, if the batch job
    repository database has not been updated with the latest
    versions of the tables.
    
    The exception stack trace will look similar to:
    
    
    java.lang.ClassCastException:
    com.ibm.jbatch.container.persistence.jpa.JobInstanceEntityV2
    incompatible with
    com.ibm.jbatch.container.persistence.jpa.JobInstanceEntityV3
    at
    com.ibm.jbatch.container.services.impl.JPAPersistenceManager
    Impl$60.call(JPAPersistenceManagerImpl.java:2923)
    at
    com.ibm.jbatch.container.services.impl.JPAPersistenceManager
    Impl$60.call(JPAPersistenceManagerImpl.java:2919)
    at
    com.ibm.jbatch.container.services.impl.JPAPersistenceManager
    Impl$TranRequest.runInNewOrExistingGlobalTran(JPAPersistence
    ManagerImpl.java:2627)
    at
    com.ibm.jbatch.container.services.impl.JPAPersistenceManager
    Impl.updateJobInstanceWithGroupNames(JPAPersistenceManagerIm
    pl.java:2934)
    at
    com.ibm.jbatch.container.ws.impl.WSJobRepositoryImpl.updateJ
    obInstanceWithGroupNames(WSJobRepositoryImpl.java:416)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
    Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
    ssorImpl.java:95)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
    thodAccessorImpl.java:55)
        at java.lang.reflect.Method.invoke(Method.java:508)
    at
    com.ibm.ws.context.service.serializable.ContextualInvocation
    Handler.invoke(ContextualInvocationHandler.java:76)
    at
    com.ibm.ws.context.service.serializable.ContextualInvocation
    Handler.invoke(ContextualInvocationHandler.java:97)
    at
    com.sun.proxy.$Proxy42.updateJobInstanceWithGroupNames(Unkno
    wn Source)
    at
    com.ibm.ws.jbatch.jms.internal.listener.BatchJmsEndpointList
    ener.handleStartRequest(BatchJmsEndpointListener.java:562)
    at
    com.ibm.ws.jbatch.jms.internal.listener.BatchJmsEndpointList
    ener.processMessage(BatchJmsEndpointListener.java:288)
    at
    com.ibm.ws.jbatch.jms.internal.listener.BatchJmsEndpointList
    ener.onMessage(BatchJmsEndpointListener.java:243)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
    Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
    ssorImpl.java:95)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
    thodAccessorImpl.java:55)
        at java.lang.reflect.Method.invoke(Method.java:508)
    at
    com.ibm.ws.jbatch.jms.internal.listener.impl.MessageEndpoint
    Handler.invokeJMSMethod(MessageEndpointHandler.java:356)
    at
    com.ibm.ws.jbatch.jms.internal.listener.impl.MessageEndpoint
    Handler.invoke(MessageEndpointHandler.java:338)
        ... more ...
    
    and, the exception will occur repeatedly, as the message is
    "rolled back" to the dispatch queue and then redelivered.
    This will happen indefinitely, unless the JMS provider is
    configured somehow to deal with a "poison message" such as
    this.
    
    This occurs when the <batchJmsExecutor> element is
    configured with a child <operationGroup> element
    configuration, and when the job repository tables have not
    been updated to include the new table definitions introduced
    in the 18.0.0.1 fix pack, along with the batch group
    security function.     Though the new table definitions
    would typically be automatically picked up in a development
    environment, a typical production environment would turn off
    this auto-create function, and so this could occur if there
    was an attempt to use the new group security feature
    introduced in 18.0.0.1.
    

Local fix

Problem summary

  • ****************************************************************
    * USERS AFFECTED:  Users of IBM WebSphere Application Server   *
    *                  Liberty Batch JMS dispatcher/executor       *
    *                  function                                    *
    ****************************************************************
    * PROBLEM DESCRIPTION: Repeated delivery of Batch job dispatch *
    *                      JMS message resulting in                *
    *                      ClassCastException each time.           *
    ****************************************************************
    * RECOMMENDATION:                                              *
    ****************************************************************
    The problem will occur when using the Liberty Batch JMS
    dispatcher/executor function with an executor configured to use
    the "group security"  function, if the batch job repository
    database has not been updated with the latest versions of the
    tables.
    
    The exception stack trace will look similar to:
    
    java.lang.ClassCastException:
    com.ibm.jbatch.container.persistence.jpa.JobInstanceEntityV2
    incompatible with
    com.ibm.jbatch.container.persistence.jpa.JobInstanceEntityV3
        at
    com.ibm.jbatch.container.services.impl.JPAPersistenceManagerImpl
    $60.call(JPAPersistenceManagerImpl.java:2923)
        at
    com.ibm.jbatch.container.services.impl.JPAPersistenceManagerImpl
    $60.call(JPAPersistenceManagerImpl.java:2919)
        at
    com.ibm.jbatch.container.services.impl.JPAPersistenceManagerImpl
    $TranRequest.runInNewOrExistingGlobalTran(JPAPersistenceManagerI
    mpl.java:2627)
        at
    com.ibm.jbatch.container.services.impl.JPAPersistenceManagerImpl
    .updateJobInstanceWithGroupNames(JPAPersistenceManagerImpl.java:
    2934)
        at
    com.ibm.jbatch.container.ws.impl.WSJobRepositoryImpl.updateJobIn
    stanceWithGroupNames(WSJobRepositoryImpl.java:416)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
    Method)
        at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessor
    Impl.java:95)
        at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethod
    AccessorImpl.java:55)
        at java.lang.reflect.Method.invoke(Method.java:508)
        at
    com.ibm.ws.context.service.serializable.ContextualInvocationHand
    ler.invoke(ContextualInvocationHandler.java:76)
        at
    com.ibm.ws.context.service.serializable.ContextualInvocationHand
    ler.invoke(ContextualInvocationHandler.java:97)
        at
    com.sun.proxy.$Proxy42.updateJobInstanceWithGroupNames(Unknown
    Source)
        at
    com.ibm.ws.jbatch.jms.internal.listener.BatchJmsEndpointListener
    .handleStartRequest(BatchJmsEndpointListener.java:562)
        at
    com.ibm.ws.jbatch.jms.internal.listener.BatchJmsEndpointListener
    .processMessage(BatchJmsEndpointListener.java:288)
        at
    com.ibm.ws.jbatch.jms.internal.listener.BatchJmsEndpointListener
    .onMessage(BatchJmsEndpointListener.java:243)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
    Method)
        at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessor
    Impl.java:95)
        at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethod
    AccessorImpl.java:55)
        at java.lang.reflect.Method.invoke(Method.java:508)
        at
    com.ibm.ws.jbatch.jms.internal.listener.impl.MessageEndpointHand
    ler.invokeJMSMethod(MessageEndpointHandler.java:356)
        at
    com.ibm.ws.jbatch.jms.internal.listener.impl.MessageEndpointHand
    ler.invoke(MessageEndpointHandler.java:338)
        ... more ...
    
    The exception will occur repeatedly, as the message is "rolled
    back" to the dispatch queue and then redelivered.   This will
    happen indefinitely, unless the JMS provider is configured
    somehow to deal with a "poison message" such as this.
    
    This occurs when the <batchJmsExecutor> element is configured
    with a child <operationGroup> element configuration, and when
    the job repository tables have not been updated to include the
    new table definitions introduced in the 18.0.0.1 fix pack, along
    with the batch group security function.
    
    Though the new table definitions would typically be
    automatically picked up in a development environment, a typical
    production environment would turn off this auto-create function,
    and so this could occur if there was an attempt to use the new
    group security feature introduced in 18.0.0.1.
    

Problem conclusion

  • The batch executor was changed to check the table version before
    attempting to apply an operation group mapping.  If the newer,
    relevant table definitions have not been applied to the job
    repository database, the operation group mapping will simply be
    ignored.
    
    In addition, the java.lang.ClassCastException was added to the
    list of exceptions upon which to consume (rather than rolling it
    back and redelivering with the same result).
    
    For further reference on the batch job repository including the
    18.0.0.1 updates, please see this technote:
    http://www.ibm.com/support/docview.wss?uid=swg22014371
    and this whitepaper:
    http://www.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP1027
    16
    
    The fix for this APAR is currently targeted for inclusion in fix
    pack 18.0.0.2.  Please refer to the Recommended Updates page for
    delivery information:
    http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980
    

Temporary fix

  • Apply the database updates mentined above.  Until that time,
    remove any <operationGroup> mapping child elements from any
    parent <batchJmsExecutor> element (the group security assignment
    will not happen anyway without the table definitions, so this is
    useless).
    

Comments

APAR Information

  • APAR number

    PI99138

  • Reported component name

    LIBERTY PROFILE

  • Reported component ID

    5724J0814

  • Reported release

    CD0

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2018-06-13

  • Closed date

    2018-06-15

  • Last modified date

    2018-06-15

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    LIBERTY PROFILE

  • Fixed component ID

    5724J0814

Applicable component levels

  • RCD0 PSY

       UP

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSEQTP","label":"WebSphere Application Server"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"CD0","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
01 December 2021