IT27477: ONSTAT -Z CAN HANG THE SERVER IF IT FINDS A BAD MUTEX

APAR status

• Closed as program error.

Error description

• It's possible for onstat -z to report a bad mutex and hang the
  server...
  
  When the problem occurs, you will see an assertion like this:
  
      18:10:01  Assert Warning: Bad mutex encountered
      18:10:01   Who: Session(858, informix@u363, 0,
  700000fb4b79d50)
                      Thread(1051, onmode_mon, 700000fb4b47a28, 1)
                      File: mt_info.c Line: 1306
      18:10:01   Results: Couldn't reset all the spin locks
      18:10:01   Action: Please notify IBM Informix Technical
  Support.
      18:10:01  Raw hex dump of stack located in
  /crash/af.8037119.rawstk
      18:10:01  Stack for thread: 1051 onmode_mon
       base: 0x0700000fb6e21000
        len:   69632
         pc: 0x00000001000948b0
        tos: 0x0700000fb6e2fda0
      state: running
         vp: 1
  
      (oninit)afstack
      (oninit)afhandler
      (oninit)afwarn_interface
      (oninit)dospinlock
      (oninit)zero_profile
      (oninit)onmode_monitor
      (oninit)th_init_initgls
      (oninit)startup
  
  Just because this assertion occurs, that does not mean the hang
  will occur.
  
  When the hang occurs, you will see a thread spinning, constantly
  running in mt_init_mutex with a stack something like:
  
      state: running
         vp: 1
  
      (oninit)poll
      (oninit)mt_spin_lock_wait
      (oninit)mt_init_mutex
      (oninit)ptalloc
      (oninit)ptbld
      (oninit)ptbuild
      (oninit)rspnbuild_sz
      (oninit)sqispnbuild
      (oninit)bldpartn
      (oninit)bldtmpfile
      (oninit)filltemp
      (oninit)scan_open
      (oninit)IPRA.$materialize_viewtmp
      (oninit)IPRA.$materialize_viewtmp
      (oninit)IPRA.$materialize_viewtmp
      (oninit)materialize_viewtmp
      (oninit)prepselect
      (oninit)open_cursor
      (oninit)sql_open
      (oninit)sq_open
      (oninit)sqmain
      (oninit)listen_verify
  
  And the onmode_mon thread will be ready to run with a stack
  comparable to:
  
      state: ready
         vp: 1
  
      (oninit)yield_processor_mvp
      (oninit)mt_yield
      (oninit)mt_exec
      (oninit)invoke_alarmf
      (oninit)afhandler
      (oninit)afwarn_interface
      (oninit)dospinlock
      (oninit)zero_profile
      (oninit)onmode_monitor
      (oninit)th_init_initgls
      (oninit)startup
  
  In this case, the thread that was running was holding a mutex
  and there is a user in the waitlist
  waiting for a very long time:
  
      12507    700000fb3c1c6b8  dbs_partn_6        89185644 0
  89185652 3742
  
  89185673 3731
  
  522      3368
  
  4553222  3013
  
  89110462 903
  
  89086555 608
  
  89086494 91
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  ****************************************************************
  * RECOMMENDATION:                                              *
  ****************************************************************
  Fixed in IDS 11.70.xC9W2.
  

Problem conclusion

• Fixed in IDS 12.10.xC11.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  INFORMIX SERVER

• Fixed component ID

  5725A3900

Applicable component levels