Topic
  • 3 replies
  • Latest Post - ‏2014-09-01T03:59:35Z by TrushkinAndrey
PradheepT
PradheepT
7 Posts

Pinned topic What could be the cause for frequent heap dumps with leak suspects as BPEL Engine and HashMap having com/ibm/bpe/database/AIIDImpl ?

‏2014-08-29T00:59:40Z |

Hi 

Recently we are encoutering consistent heap dumps with the following leak suspects. Lot of Transaction objects having BPELEngine and HashMap (1000+children) with AIIDImpl on it. What could be causing this issue. Need help on understanding the possible causes of this behavior.

1,160,736,232 (62.46%) [208] 18 array of java/util/Hashtable$Entry 0xa43271f8
|- 549,994,520 (29.6%) [32] 2 java/util/Hashtable$Entry 0x92c4d458
   |- 549,994,440 (29.6%) [104] 10 com/ibm/ws/activity/LocalActivityContext 0x92c4d478
      |- 549,993,984 (29.6%) [32] 1 java/util/Vector 0x92c58920
         |- 549,993,952 (29.6%) [56] 1 array of java/lang/Object 0x9550d728
            |- 549,993,896 (29.6%) [136] 11 com/ibm/ws/util/ThreadPool$Worker 0xa5c656e0
               |- 548,640,008 (29.52%) [304] 18 array of java/lang/Object 0xa417b5f0
                  |- 548,629,512 (29.52%) [32] 3 com/ibm/ws/tx/jta/TranManagerImpl 0xa417b828
                     |- 548,629,480 (29.52%) [256] 10 com/ibm/ws/tx/jta/TransactionImpl 0x92c51770
                        |- 548,626,920 (29.52%) [40] 2 com/ibm/ws/tx/jta/RegisteredSyncs 0x95444218
                           |- 548,626,880 (29.52%) [32] 2 array of java/util/ArrayList 0x95444240
                              |- 548,626,560 (29.52%) [32] 1 java/util/ArrayList 0x95444280
                                 |- 548,626,528 (29.52%) [56] 2 array of java/lang/Object 0x954442d8
                                    |- 548,625,832 (29.52%) [24] 2 com/ibm/ws/jtaextensions/SynchronizationCallbackWrapper 0x95444328
                                       |- 548,625,712 (29.52%) [24] 2 com/ibm/bpe/database/helper/TomSynchronizationCallback 0x95444408
                                          |- 548,625,576 (29.52%) [424] 90 com/ibm/bpe/database/TomExtended 0xa370bb90
                                             |- 449,385,736 (24.18%) [32] 1 java/util/ArrayList 0xa164a048
                                                |- 449,385,704 (24.18%) [72] 3 array of java/lang/Object 0xa164a068
                                                   |- 449,383,152 (24.18%) [200] 18 com/ibm/bpe/engine/BpelEngine 0xa4325718
                                                      |- 305,830,800 (16.46%) [16] 1 java/util/HashSet 0x91616ae8
                                                         |- 305,830,784 (16.46%) [48] 2 java/util/HashMap 0x91057820
                                                            |- 305,830,720 (16.46%) [33,554,448] 1,820,250 array of java/util/HashMap$Entry 0xdac16610
                                                               |- 1,368 (0%) [32] 3 java/util/HashMap$Entry 0xdeda4b18
                                                                  |- 1,296 (0%) [32] 3 java/util/HashMap$Entry 0xe19ecfd0
                                                                     |- 1,224 (0%) [32] 3 java/util/HashMap$Entry 0xdf2c8a20
                                                                     |- 40 (0%) [40] 0 com/ibm/bpe/database/AIIDimpl 0xdf2c89f8
 

 

  • TrushkinAndrey
    TrushkinAndrey
    72 Posts

    Re: What could be the cause for frequent heap dumps with leak suspects as BPEL Engine and HashMap having com/ibm/bpe/database/AIIDImpl ?

    ‏2014-08-29T04:21:21Z  

    Hi.

     

    Try to check your transaction logic in your BPEL: how many transactions with state saving you have in your process?

  • PradheepT
    PradheepT
    7 Posts

    Re: What could be the cause for frequent heap dumps with leak suspects as BPEL Engine and HashMap having com/ibm/bpe/database/AIIDImpl ?

    ‏2014-08-29T16:42:50Z  

    Hi.

     

    Try to check your transaction logic in your BPEL: how many transactions with state saving you have in your process?

    This happens on un-usual load, and We have more than 30+ long running processes and approx 10-20 microflows in the system,

    Would it be feasible to enable some tracing to get to know which process is having or costing more transactions ? Are there any options to do so ? 

     

  • TrushkinAndrey
    TrushkinAndrey
    72 Posts

    Re: What could be the cause for frequent heap dumps with leak suspects as BPEL Engine and HashMap having com/ibm/bpe/database/AIIDImpl ?

    ‏2014-09-01T03:59:35Z  
    • PradheepT
    • ‏2014-08-29T16:42:50Z

    This happens on un-usual load, and We have more than 30+ long running processes and approx 10-20 microflows in the system,

    Would it be feasible to enable some tracing to get to know which process is having or costing more transactions ? Are there any options to do so ? 

     

    Hi.

     

    I think, that you should check design of your system: transaction boundaries in BPEL, service invocation methods in microflow, transaction qualifiers of the microflows, ... Because WebSphere has some wonderful property: tracing in most cases produce more load then your processes.

     

    For example, you can try to reduce transactions count in your BPEL processes where you can, because all new transactions need in save there states with all variable values in DB. And you can check variables in BPEL process state. May be some of them are duplicated - you should avoid this.