LO63407: CUSTOMERS DOMINO SERVER CRASH ON HANDLEDEREFERENCE - PANIC: OBJECT HANDLE IS INVALID WHILE OPENING A COLLECTION

APAR status

• Closed as unreproducible.

Error description

• NSD REVIEW nsd_W32I_ABCACME_2011_08_30@11_45_48
  
  1) The first thing that I do when analyzing an NSD is look at
  the Name of the Server, Date & Time, OS Version and Notes
  Version.
  
  Host Name       : ABCACME
  User Name       : SYSTEM
  Date            : Tue Aug 30 11:45:48 2011
  Windows Dir     : C:\Windows
  Arguments       : "C:\Program Files
  (x86)\IBM\Lotus\Domino\nsd.exe" -dumpandkill -termstatus 1
  -panicdirect -crashpid 3504 -crashtid 3912 -runtime 900
  NSD Version     : 8.5.22.1048 (Release 8.5.2FP2)
  OS Version      : Windows/2008 6.0 [64-bit] (Build 6002),
  PlatID=2, Service Pack 2 (8 Processors)
     Running as 32-bit Windows application on 64-bit Windows
  Build time      : Wed Mar 23 02:53:24 2011
  Latest file mod : Fri Feb 18 11:33:30 2011
  Domino Version   : Release 8.5.2FP2 HF482 (32-bit server)
  Keyview Version : 10.8.0.0
  
  2) Once we have verified the Build  I then search for "OS
  Process" to see what is running on the server, when the OS was
  rebooted and we can see when the server crashed outlined in RED.
  
  3) Now that we have reviewed the OS process we want to see what
  task the server crash on so we go to the top of the NSD and
  search for the word "FATAL" which will bring us down to the
  fatal thread. We then review the fatal thread to see if it a
  known issue and we continue our search this time searching using
  the following: "0db0:  0f48"
  
  ############################################################
  ### thread 33/141: [ nserver:  0db0:  0f48] FATAL THREAD (Panic)
  ### FP=0x0be4dd98, PC=0x7753938d, SP=0x0be4dd2c
  ### stkbase=0x0be50000, total stksize=262144, used stksize=8916
  ### EAX=0x00000000, EBX=0x00000000, ECX=0x00000000,
  EDX=0x00000000
  ### ESI=0x0be4dd70, EDI=0x00000000, CS=0x00000023, SS=0x0000002b
  ### DS=0x0000002b, ES=0x0000002b, FS=0x00000053, GS=0x0000002b
  Flags=0x00000206
  ############################################################
   [ 1] 0x7753938d ntdll.ZwWaitForSingleObject+21
  (5ff4,dbba0,0,be4e320)
   [ 2] 0x769a11d8 kernel32.WaitForSingleObject+18
  (5ff4,dbba0,0,be4e53c)
  @[ 3] 0x6020d53d nnotes.OSRunExternalScript@8+1309 (384,1)
  @[ 4] 0x6020d9bf nnotes.FRTerminateWindowsResources+975
  (1,1010,1,0)
  @[ 5] 0x6020dde8 nnotes.OSFaultCleanupExt@24+984
  (27b6a68,1010,0,0,0,be4e864)
  @[ 6] 0x6020de6a nnotes.OSFaultCleanup@12+26 (0,1010,0)
  @[ 7] 0x60219704 nnotes.OSNTUnhandledExceptionFilter@4+276
  (be4f89c)
  @[ 8] 0x601dae4d nnotes.Panic@4+589 (60eda5f4)
  @[ 9] 0x601daf4c nnotes.Halt@4+28 (2b4)
  @[10] 0x6000b915 nnotes.HANDLEDereference@4+117 (6d5)
  @[11] 0x6000b80e nnotes.InitDbContextExt@12+238 (6d5,be4f914,3)
  @[12] 0x6000b942 nnotes.InitDbContext@8+18 (6d5,be4f914)
  @[13] 0x6006c2fe nnotes.NSFDbMaintainsUnreads@8+30
  (6d5,585c6fad)
  @[14] 0x10042c42 nserverl.ServerOpenCollection@8+802
  (58a95120,3df800b3)
  @[15] 0x10021c63 nserverl.DbServer@8+2515 (7f690242,9658007c)
  @[16] 0x10037277 nserverl.WorkThreadTask@8+1655 (7b68f8c,0)
  @[17] 0x100018ce nserverl.Scheduler@4+750 (0)
  @[18] 0x6015577f nnotes.ThreadWrapper@4+175 (0)
   [19] 0x76a1eccb kernel32.BaseThreadInitThunk+14
  (0,7cb8fde6,0,0)
   [20] 0x7756d80d ntdll.RtlCreateUserProcess+140 (601556d0,0,0,0)
   [21] 0x7756da1f ntdll.RtlCreateProcessParameters+78 (0,0,0,0)
  
  4) After a couple of passes through the Fatal Thread stack
  frames we continue Searching on  "0db0:  0f48" and search until
  we reach the MM/OS section
  
  5)  Once we reach the MM/OS section we can see when the server
  was started and again we see the reference to the FATAL Stack :
  "0db0:  0f48" and continue to search the NSD until we hit the
  Vthread Mapped to Pthread.
  
  <@@ ------ Notes Data -> OS Data -> MM/OS Structure Information
  (Time 11:47:02) ------ @@>
  
   Start Time = 08/27/2011 06:17:57 PM
   Crash Time = 08/30/2011 11:45:47 AM
   Domino Controller Enabled = 1
   Domino Controller Current Log = dcntrlr201108300000.log
   Error Message = PANIC: Object handle is invalid
   SharedDPoolSize = 4194304
   FaultRecovery = 0x00010010
   Cleanup Script Timeout= 900
   Crash Limits = 3 crashes in 5 minutes
   StaticHang = Virtual Thread [ nserver:  0db0:  0267] (Native
  thread [ nserver:  0db0:  0f48]) (0xdb0/0x267/0xf48)
   ConfigFileSem =  (  SEM:#0:0x010d) n=0, wcnt=-1, Users=-1,
  Owner=[        :  0000]
   FDSem         =  ( RWSEM:#52:0x410f) rdcnt=-1, refcnt=1
  Writer=[   nHTTP:  1608:  1920], n=52, wcnt=0, Users=0,  Owner=[
  nHTTP:  1608:  1920]
  
  6.) Now that we have reached the Vthread mapped to the Pthread
  we can see 1 database. This being mail\y\abc.nsf which shows
  2 views open in this database at the time of the crash and this
  is the database I would focus on. It would be suggested to pull
  a new replica of the database below and run complete offline
  maintenance (nFixup -F -J, nCompact -b, nUpdall -R) on this
  database if you continue to see the crash with this database or
  if not transactionally logged (nFixup -F, nCompact -c,
  nUpdall -R)
  
  **  VThread [ nserver:  0db0:  0267]
  .Mapped To: PThread [ nserver:  0db0:  0f48]
  .Description: Test User in OPEN_COLLECTION
  ..     using: Primal Thread [ nserver:  0db0:  0026]
  ..      SOBJ: addr=0x3a4f4dac, h=0xf010336b t=0xc30a
  (BLK_LOOKUP_THREAD)
  ..      SOBJ: addr=0x167fb60c, h=0xf010336a t=0xc130 (BLK_TLA)
  ..      SOBJ: addr=0x0017c834, h=0xf0103374 t=0xcf02
  (BLK_FT_STATIC)
  ..      SOBJ: addr=0x4cfa40b4, h=0xf010336c t=0xc275 (BLK_NSFT)
  ..  Database: D:\notes\data\mail\y\ABC.nsf
  ....       DBH:  10466, By: CN=Test User/OU=ST/OU=MG/O=MGC,
  WasAccessed=Yes
  ......      view: hCol=10478, cg=N noteID=1954, sessID: [693:
  1255] (MiniView - Followup2)|MiniView - Followup2
  ....       DBH:  10462, By: CN=Test User/OU=ST/OU=MG/O=MGC,
  WasAccessed=Yes
  ....       DBH:  10695, By: CN=Test User/OU=ST/OU=MG/O=MGC,
  WasAccessed=Yes
  ......      view: hCol=10745, cg=N noteID=4562, sessID: [693:
  1255] ($Inbox)
  ....       DBH:  10675, By: CN=Test User/OU=ST/OU=MG/O=MGC,
  WasAccessed=Yes
  

Local fix

• Suggested to pull a new replica of the database in question and
  complete the following offline maintenence:
  
  Translogged Servers:
  (nFixup -F -J, nCompact -b, nUpdall -R)
  
  or
  
  non Translogged Servers:
  
  (nFixup -F, nCompact -c, nUpdall -R)
  

Problem summary

Problem conclusion

Temporary fix

Comments

• This APAR is associated with SPR# DADS8L8HW5.
  Fixed in 9.0/8.5.3 FP5
  

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

Applicable component levels

• R852 PSN

     UP