IBM Support

LO68746: HTTP CRASH PANIC: SEMAPHORE INVALID OR NOT ALLOCATED

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as fixed if next.

Error description

  • Error Description￘
nsd_W64I_ACME SERVER_2012_03_23@13_12_06.log

Host Name       : ACME SERVER
User Name       : SYSTEM
Date            : Fri Mar 23 13:12:06 2012
Windows Dir     : C:\Windows
Arguments       : "P:\Program Files\IBM\Lotus\Domino\nsd.exe"
-dumpandkill -termstatus 1 -panicdirect -crashpid 5796
-crashtid 4140 -runtime 600 -ini "P:\Program
Files\IBM\Lotus\Domino\notes.ini" -svcreq 128
NSD Version     : 8.5.23.1132 (Release 8.5.2FP3)
OS Version      : Windows/2008 R2 6.1 [64-bit] (Build 7601),
PlatID=2, Service Pack 1 (2 Processors)
Build time      : Mon Jul 11 03:52:14 2011
Latest file mod : Fri May 13 09:03:31 2011
Domino Version   : Release 8.5.2FP3 (64-bit server)
Keyview Version : 10.8.0.0


############################################################
### thread 20/70: [   nHTTP:  16a4:  102c] FATAL THREAD (Panic)
### FP=0x6de07188, PC=0x77cb135a, SP=0x6de07188
### stkbase=0x6de10000, total stksize=4194304, used
stksize=36472
### EAX=0x00000041, EBX=0x00000000, ECX=0x6de06162,
EDX=0x7FE9017D537
### ESI=0x000927c0, EDI=0x00001f9c, CS=0x00000033,
SS=0x7FE0000002B
### DS=0x00000000, ES=0x00000000, FS=0x00000000,
GS=0x7FE00000000 Flags=0x00000287
############################################################
 [ 1] 0x77cb135a ntdll.ZwWaitForSingleObject+10
 [ 2] 0x7FEFE1B10DC KERNELBASE.WaitForSingleObjectEx+156
@[ 3] 0x00e6535f nnotes.FRSendCommandToService+1931
@[ 4] 0x00e68b73 nnotes.OSRunExternalScript+5835
@[ 5] 0x00e624d5 nnotes.FRTerminateWindowsResources+2277
@[ 6] 0x00e6bbe2 nnotes.OSFaultCleanupExt+622
@[ 7] 0x00e6c609 nnotes.OSFaultCleanup+29 (0,0,0,0)
@[ 8] 0x00e8665d nnotes.OSNTUnhandledExceptionFilter+677
@[ 9] 0x00df85ca nnotes.Panic+682 (0,6,6de0fe08,e2e156)
@[10] 0x00e2cf87 nnotes.OSUnlockSem+35
@[11] 0x00e2675b nnotes.LockMemHandle+371
@[12] 0x00e27ad4 nnotes.OSMemoryLock+120 (6de0fda0,0,0,6cc33750)
@[13] 0x02900cc7 nnotes.DbBktDescGet+2391

4) After a couple of passes through the Fatal Thread stack
frames we continue Searching on  "16a4:  102c" and search until
we reach the MM/OS section

5)  Once we reach the MM/OS section we can see when the server
was started and again we see the reference to the FATAL Stack :
 "16a4:  102c" and continue to search the NSD until we hit the
Vthread Mapped to Pthread.

<@@ ------ Notes Data -> OS Data -> MM/OS Structure Information
(Time 13:12:53) ------ @@>

 Start Time = 03/18/2012 05:09:04 AM
 Crash Time = 03/23/2012 01:12:06 PM
 Domino Controller Enabled = 1
 Domino Controller Current Log = dcntrlr201203230000.log
 Error Message = PANIC: semaphore invalid or not allocated
 Console Position = 163423551
 SharedDPoolSize = 33554432
 FaultRecovery = 0x00010010
 Cleanup Script Timeout= 600
 Crash Limits = 3 crashes in 5 minutes
 StaticHang = Virtual Thread [   nHTTP:  16a4:  000c]
(Native thread [   nHTTP:  16a4:  102c]) (0x16a4/0xc/0x102c)
 ConfigFileSem =  (  SEM:#0:0x010d) n=0, wcnt=-1, Users=-1,
Owner=[        :  0000]
 FDSem         =  ( RWSEM:#53:0x410f) rdcnt=-1, refcnt=0
Writer=[        :  0000], n=53, wcnt=-1, Users=0,  Owner=[
  :  0000]

A Review of the first HTTP Thread has 720 lines which
caused the stack to run out of room which would result in crash:

############################################################
### thread 21/70: [   nHTTP:  16a4:  130c]
### FP=0x6de21c78, PC=0x77cb165a, SP=0x6de21c78
### stkbase=0x6e210000, total stksize=4194304, used
stksize=4121480
############################################################
 [ 1] 0x77cb165a ntdll.ZwDelayExecution+10
(6de21d38,7fbea20,668c99e0,e1b009)
 [ 2] 0x7FEFE1B1203 KERNELBASE.SleepEx+179 (1,0,0,0)
@[ 3] 0x00e0911e nnotes.OSDelayThread+50 (6de227c0,b018000c,0,0)
@[ 4] 0x00e03816 nnotes.StaticThreadMem+722 (0,0,0,e2dac9)
@[ 5] 0x00e044fd nnotes.OSStaticMem+3081
@[ 6] 0x00e1d64f nnotes.OSGetSignalHandler+23
@[ 7] 0x02efdb0b nnotes.Endp_DefaultAbortCheck+43
@[ 8] 0x02efda9d nnotes.Endp_DoAbortCheck+277
@[ 9] 0x003d79b4 nNTCP.cmd_poll+2076
@[10] 0x02f04acc nnotes.nti_poll+292 (2,7188014c,8d940a0,0)
@[11] 0x02ef8790 nnotes.PortDriver_ReceiveWait+184
@[12] 0x02ef249e nnotes.PortDriver_WaitHelper+210
@[13] 0x02ef2aba nnotes.PortDriver_WaitExtended+186
@[14] 0x022dbfa5 nnotes.NetWaitTimeout+1277
@[15] 0x01d30dc4 nnotes.ReceiveNetBuffer+276
@[16] 0x01d31a0f nnotes.SendBufferGetResponse+527
@[17] 0x01d95aca nnotes.ClientFindByKeyExtended2+954
@[18] 0x02c25b0b nnotes.NIFFindByKeyExtended2+279
@[19] 0x02be0577 nnotes.LookupNameWithRead+331
@[20] 0x02bd99ee nnotes.ReturnNameInfo+810
@[21] 0x02bd1a87 nnotes.Lookup+11635 (125,e0a7e6,15c,125)
@[22] 0x02b631c6 nnotes.NAMESwitchToNLThread+830 (0,0,0,0)
@[23] 0x02bbf479 nnotes.ThreadedNAMELookup2+205
@[24] 0x02bbf303 nnotes.NAMELookupExtended+635
@[25] 0x009a8be5 ninotes.ShimNameResolveObj::nameLookup+305
@[26] 0x009a8a95ninotes.ShimNameResolveObj::nameLookupAtOnce
@[27] 0x009a5e38ninotes.ShimNameResolveObj::enumerateEntry
@[28] 0x0088e4fd
ninotes.UltraliteSendMail2Command::ExpandAndValidateOn eItem

[Repeating Lines]

@[698] 0x0088f230
ninotes.UltraliteSendMail2Command::ExpandAndValidateOneItem
@[699] 0x0088db19
ninotes.UltraliteSendMail2Command::ExpandAndValidateRecipients
@[700] 0x0088c7d7 ninotes.UltraliteSendMail2Cmd+251
@[701] 0x007e22be
ninotes.HaikuCommandCallList::DispatchCommand+2426 (0,0,0,0)
@[702] 0x007c6f7f ninotes.Haiku::ProcessULEditAction+447
@[703] 0x007c4032 ninotes.Haiku::HandleDominoCmd+954
@[704] 0x00722ea4 ninotes.Haiku::HandleCmd+1264
@[705] 0x00495e5a ninotes.CmdHandlerBase::PrivHandle+146
@[706] 0x00491db7 ninotes.CmdHandler::PrivHandle+187
@[707] 0x00491f4d ninotes.CmdHandler::Handler+221
@[708] 0x0047a8e7 ninotes.Cmd::Execute+35
@[709] 0x0051a745 ninotes.InotesHTTPProcessRequestImpl+2097
@[710] 0x00519ee3 ninotes.InotesHTTPProcessRequest+23
@[711] 0x10029042
nhttpstack.HTInotesRequest::ProcessRequest+142
@[712] 0x10020116
nhttpstack.HTRequestExtContainer::ProcessRequest+942
@[713] 0x1003b9b9 nhttpstack.HTRequest::ProcessRequest+6217
@[714] 0x1004857b nhttpstack.HTSession::StartRequest+2723
@[715] 0x10059942 nhttpstack.HTWorkerThread::CheckForWork+914
(6cc5cfa8,10059f24,FFFFFFFFFFFFFFE,e2e1d7)
@[716] 0x10059521 nhttpstack.HTWorkerThread::ThreadMain+81
@[717] 0x1005313b nhttpstack.HTThreadBeginProc+67 (0,0,0,0)
@[718] 0x00e093a2 nnotes.ThreadWrapper+330 (0,0,0,0)
 [719] 0x77a5652d kernel32.BaseThreadInitThunk+13 (0,0,0,0)
 [720] 0x77c8c521 ntdll.RtlUserThreadStart+33 (0,0,0,0)

8.) A Review of the second HTTP Thread has 364 lines and the
same type of function calls and a very large stack:

############################################################
### thread 23/70: [   nHTTP:  16a4:  0124]
### FP=0x6e830d38, PC=0x77cb165a, SP=0x6e830d38
### stkbase=0x6ea10000, total stksize=4194304, used
stksize=1962696
############################################################
 [ 1] 0x77cb165a ntdll.ZwDelayExecution+10
(6e830df8,e0a7e6,0,e2cf94)
 [ 2] 0x7FEFE1B1203 KERNELBASE.SleepEx+179 (1,0,0,0)
@[ 3] 0x00e0911e nnotes.OSDelayThread+50 (6e831880,b018000c,0,0)
@[ 4] 0x00e03816 nnotes.StaticThreadMem+722 (0,0,0,e2dac9)
@[ 5] 0x00e044fd nnotes.OSStaticMem+3081
@[ 6] 0x00e1d64f nnotes.OSGetSignalHandler+23
@[ 7] 0x02efdb0b nnotes.Endp_DefaultAbortCheck+43
@[ 8] 0x02efda9d nnotes.Endp_DoAbortCheck+277
@[ 9] 0x003d79b4 nNTCP.cmd_poll+2076
@[10] 0x02f04acc nnotes.nti_poll+292 (2,f1c00151,8d940a0,0)
@[11] 0x02ef8790 nnotes.PortDriver_ReceiveWait+184
@[12] 0x02ef249e nnotes.PortDriver_WaitHelper+210
@[13] 0x02ef2aba nnotes.PortDriver_WaitExtended+186
@[14] 0x022dbfa5 nnotes.NetWaitTimeout+1277
@[15] 0x01d30dc4 nnotes.ReceiveNetBuffer+276
@[16] 0x01d31a0f nnotes.SendBufferGetResponse+527
@[17] 0x01d95aca nnotes.ClientFindByKeyExtended2+954
@[18] 0x02c25b0b nnotes.NIFFindByKeyExtended2+279
@[19] 0x02be0577 nnotes.LookupNameWithRead+331
@[20] 0x02bd99ee nnotes.ReturnNameInfo+810
@[21] 0x02bd1a87 nnotes.Lookup+11635 (5e9,e0a7e6,15c,5e9)
@[22] 0x02b631c6 nnotes.NAMESwitchToNLThread+830 (0,0,0,0)
@[23] 0x02bbf479 nnotes.ThreadedNAMELookup2+205
@[24] 0x02bbf303 nnotes.NAMELookupExtended+635
@[25] 0x009a8be5 ninotes.ShimNameResolveObj::nameLookup+305
@[26] 0x009a8a95
ninotes.ShimNameResolveObj::nameLookupAtOnce+349
@[27] 0x009a5e38
ninotes.ShimNameResolveObj::enumerateEntry+2124
@[28] 0x0088e4fd
ninotes.UltraliteSendMail2Command::ExpandAndValidateOneItem+1761

@[342] 0x0088f230
ninotes.UltraliteSendMail2Command::ExpandAndValidateOneItem+5140
@[343] 0x0088db19
ninotes.UltraliteSendMail2Command::ExpandAndValidateRecipients+6
@[344] 0x0088c7d7 ninotes.UltraliteSendMail2Cmd+251
@[345] 0x007e22be
ninotes.HaikuCommandCallList::DispatchCommand+2426 (0,0,0,0)
@[346] 0x007c6f7f ninotes.Haiku::ProcessULEditAction+447
@[347] 0x007c4032 ninotes.Haiku::HandleDominoCmd+954
@[348] 0x00722ea4 ninotes.Haiku::HandleCmd+1264
(864fe270,2013978,62579CA00638738,8cd2adb1)
@[349] 0x00495e5a ninotes.CmdHandlerBase::PrivHandle+146
@[350] 0x00491db7 ninotes.CmdHandler::PrivHandle+187
@[351] 0x00491f4d ninotes.CmdHandler::Handler+221
@[352] 0x0047a8e7 ninotes.Cmd::Execute+35
@[353] 0x0051a745 ninotes.InotesHTTPProcessRequestImpl+2097
@[354] 0x00519ee3 ninotes.InotesHTTPProcessRequest+23
@[355] 0x10029042
nhttpstack.HTInotesRequest::ProcessRequest+142
(0,10037f00,6ea0e630,8c19e288)
@[356] 0x10020116
nhttpstack.HTRequestExtContainer::ProcessRequest+942
@[357] 0x1003b9b9 nhttpstack.HTRequest::ProcessRequest+6217
@[358] 0x1004857b nhttpstack.HTSession::StartRequest+2723
@[359] 0x10059942 nhttpstack.HTWorkerThread::CheckForWork+914
(6cc5e728,10059f24,FFFFFFFFFFFFFFE,e2e1d7)
@[360] 0x10059521 nhttpstack.HTWorkerThread::ThreadMain+81
@[361] 0x1005313b nhttpstack.HTThreadBeginProc+67 (0,0,0,0)
@[362] 0x00e093a2 nnotes.ThreadWrapper+330 (0,0,0,0)
 [363] 0x77a5652d kernel32.BaseThreadInitThunk+13 (0,0,0,0)
 [364] 0x77c8c521 ntdll.RtlUserThreadStart+33 (0,0,0,0)

Local fix

  • Local Fix&#65496;
NA

Problem summary

  • This APAR is closed as FIN. We have deferred the fix to a
 future release.

Problem conclusion

  • 



Temporary fix


    

  • 



Comments


    

  • This APAR is associated with SPR# DADS8TASDE.
This APAR is closed as FIN. We have deferred the fix to a
 future release.

    



APAR Information




    

  • APAR number
    LO68746
    • 

  • Reported component name
    DOMINO SERVER
    • 

  • Reported component ID
    5724E6200
    • 

  • Reported release
    852
    • 

  • Status
    CLOSED  FIN
    • 

  • PE
    NoPE
    • 

  • HIPER
    NoHIPER
    • 

  • Special Attention
    NoSpecatt
    • 

  • Submitted date
    2012-04-12
    • 

  • Closed date
    2012-05-15
    • 

  • Last modified date
    2012-05-15
    • 





    

  • APAR is sysrouted FROM one or more of the following:
    

    • 

  • APAR is sysrouted TO one or more of the following:
    

    • 



Fix information


    



Applicable component levels


    

  • R852  PSN
       UP
    • 








      
              
                            

              

              [{"Business Unit":{"code":"BU055","label":"Cognitive Applications"},"Product":{"code":"SSKTMJ","label":"Lotus Domino"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"8.5.2","Edition":"","Line of Business":{"code":"","label":""}}]
              

                          

          
 
        

      

    

      

        

          

            
Document Information


            

            


                        

            Modified date:
            

            15 May 2012
            

            
                      

        


        

        


      

    

  



    

  

  
    
            

          
Page Feedback