Topic
6 replies Latest Post - ‏2010-11-04T01:53:47Z by SystemAdmin
eyley
eyley
6 Posts
ACCEPTED ANSWER

Pinned topic Region crash when using multi-region environments. - Need any help...

‏2010-08-13T01:47:40Z |
I have made 13 regions on AIX 6.1. TL5.
TXseries version is 7.1 and fixpack 7.1.0.2 applied.

Till 4 regions, no crash.
But serveral decades of minutes passed after starting all 13 regions, some of regions were crashed. Now only 9 regions are alive, 4 regions gone.

The message is
========================
ERZ057002E/0136 08/12/10 18:19:08.094028572 F81TAR21 12845234/0001 : CICS i
nternal error: Unsuccessful condition '(!LogInUse)' for function 'DamJO_LogDaemo
n ' on line 410 in file '/cics/FSB/src/dam/jo/src/damjo_logd.c' of module 24
ERZ057005E/0136 08/12/10 18:19:08.098259009 F81TAR21 12845234/0001 : CICS i
nternal error: Abnormally terminating region with exit code 'U5701'
ERZ010003I/0094 08/12/10 18:19:08.098429623 F81TAR21 12845234/0001 : CICS i
s performing region abnormal termination in process 'cicsld'
================================
And dumps was created and regions gone.

Anyone can talk about LogDaemon?
What actions needed?

I just requested the PMR process, but need more speedy answer.
Anybody who can help me!! Please...
Updated on 2010-11-04T01:53:47Z at 2010-11-04T01:53:47Z by SystemAdmin
  • SystemAdmin
    SystemAdmin
    308 Posts
    ACCEPTED ANSWER

    Re: Region crash when using multi-region environments. - Need any help...

    ‏2010-08-13T05:02:11Z  in response to eyley
    Unfortunately the error information you have pasted in the post is too little to understand the problem. Would you be able to attach the console.<nnnnnn> and symrecs.<nnnnnn> files in this forum?. In general the logs are used by the application servers to store the transactional records and these logs are managed by the LogDaemon. There would be multiple logs (which you would find under the region's log directory) as there will be multiple application servers. Briefly, from the error it does look like that the application server is not able to open the log file, as it is already in use. Usually the logs are identified by the Application server ID (which starts from 101, 102, 103, ... and so on). So one way to check is to see in the console log if there are multiple application servers started with the same Application Server Id (a situation that shouldn't occur actually!!!)...then this can result in this error.

    In any case if you attach the console log, I can take a quick look.

    Cheers,
    Hari
    • eyley
      eyley
      6 Posts
      ACCEPTED ANSWER

      Re: Region crash when using multi-region environments. - Need any help...

      ‏2010-08-13T05:43:44Z  in response to SystemAdmin
      Hi Hari,
      Thank you for keen attention.

      in RD, min server=1, max server=30.

      ========================
      :ml14
      ResourceDescription="Region Definition"
      AmendCounter=15
      Modifiable=yes
      StartType=cold
      Groups=
      StartupProgList=""
      ShutdownProgList1=""
      ShutdownProgList2=""
      DefaultUserId="CICSUSER"
      FileSystemType=SFS
      RDBMSInstance=""
      FileRSLCheck=external
      TransientDataRSLCheck=external
      TemporaryStorageRSLCheck=external
      JournalRSLCheck=external
      ProgramRSLCheck=external
      TransactionRSLCheck=external
      ESMLoad=no
      ESMModule=""
      ReleaseNum="0710"
      LocalSysId="FA15"
      LocalNetworkName="KRIBKKB1"
      CWASize=512
      MinServer=1
      MaxServer=30
      ClassMaxTasks=1,1,1,1,1,1,1,1,1,1
      ClassMaxTaskLim=0,0,0,0,0,0,0,0,0,0
      ServerIdleLimit=300
      SafetyLevel=none
      RPCListenerThreads=0
      DateForm=ddmmyy
      ClassTableSize=5,50,50,1,50,50,50,20,5,50,1,20,5,5,20
      MaxRegionPool=31457280
      MaxTaskPrivatePool=10485760
      MaxTSHPool=10485760
      RegionPoolThreshold=90
      TaskShPoolThreshold=90
      TaskSHNumBuckets=512
      LoadDataNumBuckets=512
      SysDump=no
      PCDump=yes
      ABDump=yes
      DumpName="dumps"
      CoreDumpName="dir1"
      TraceDirectorySystem="/var/cics_regions/%R/dumps/dir1"
      TraceDirectoryUser="/var/cics_servers/archives"
      TraceFlagAux=off
      TraceFlagBuffer=on
      TraceFlagExec=off
      TraceFlagExternal=off
      TraceFlagMaster=off
      TraceFlagSystem=off
      TraceFlagUser=off
      TraceMaxSizeAux=0
      TraceMaxSizeBuffer=131072
      TraceSystemSpec="all=4"
      IntrospectInterval=30
      IntrospectLevel=minimal
      ISCDelayMinutes=30
      CUBSDelayMinutes=30
      CARPDelayHours=8
      ProtectPurgeDelayPeriod=8
      PurgeDelayPeriod=8
      StatsRecord=yes
      StatFile="statsfile"
      SufficesSupported=yes
      CheckpointInterval=10000
      DefaultFileServer="/.:/cics/sfs/upafep81"
      RecTSQFile="F81TAR21cicsrectsqfile"
      RecTSQIndex="cicsrectsqidx"
      RecTSQVol="sfs_%S"
      RecTSQPrePages=5
      RecTSQMaxRecs=1000000
      NonRecTSQFile="F81TAR21cicsnrectsqfil"
      NonRecTSQIndex="cicsnrectsqidx"
      NonRecTSQVol="sfs_%S"
      NonRecTSQPrePages=5
      NonRecTSQMaxRecs=1000000
      LogicalTDQFile="F81TAR21cicstdqlgfile"
      LogicalTDQIndex="cicstdqlgidx"
      LogicalTDQVol="sfs_%S"
      LogicalTDQPrePages=5
      LogicalTDQMaxRecs=1000000
      PhysicalTDQFile="F81TAR21cicstdqphfile"
      PhysicalTDQIndex="cicstdqphidx"
      PhysicalTDQVol="sfs_%S"
      PhysicalTDQPrePages=5
      PhysicalTDQMaxRecs=1000000
      NonRecTDQFile="F81TAR21cicstdqnofile"
      NonRecTDQIndex="cicstdqnoidx"
      NonRecTDQVol="sfs_%S"
      NonRecTDQPrePages=5
      NonRecTDQMaxRecs=1000000
      LocalQProtectFile="F81TAR21cicsplqfile"
      LocalQProtectIndex="cicsplqidx"
      LocalQProtectVol="sfs_%S"
      LocalQProtectPrePages=5
      LocalQProtectMaxRecs=1000000
      LocalQFile="F81TAR21cicsnlqfile"
      LocalQIndex="cicsnlqidx"
      LocalQVol="sfs_%S"
      LocalQPrePages=5
      LocalQMaxRecs=1000000
      TSQAgeLimit=20
      ProgramCacheSize=0
      LocalLUName="TFEP15"
      RegionPoolBase=2684354560
      TaskSharedPoolBase=2952790016
      ServerSideTran=no
      AllowDebugging=no
      HTMLBrowser=""
      AutoinstallMode=default
      MaxConsoleSize=0
      MaxTaskCPU=0
      MaxTaskCPUAction=warning
      TransDumpTrace=no
      PostScanInterval=0
      IIOPObjrefsVersion="1.1"
      IIOPRequestTimeout=100
      IIOPGroupUUID=""
      IIOPGroupHost=""
      IIOPGroupPort=""
      IIOPGroupSSLPort=""
      XPRecvTimeout=0
      TDPurgeInterval=0
      EAMLoad=no
      EAMModule=""
      ServerMemCheckInterval=0
      ServerMemCheckLimit=0
      WLMListenerList="{0.0.0.0,0}"
      WLMMinServer="{0}"
      WLMMaxServer="{0}"
      WLMWillingness="{-1}"
      WLMResponseTimeOut=10
      EnableAppProbe=no
      EnableTaskInfo=no
      DumpRemoteInfo=no
      =============================

      log directory hold 101 ~ 104 AS content and control files.
      Are those necessary also?

      Thank you again, Hari!
    • eyley
      eyley
      6 Posts
      ACCEPTED ANSWER

      Re: Region crash when using multi-region environments. - Need any help...

      ‏2010-08-13T06:21:01Z  in response to SystemAdmin
      Hari,

      As you said, I saw the cicsas 101 were started twice in console message.
      It may be problem?

      Thanks,
      • SystemAdmin
        SystemAdmin
        308 Posts
        ACCEPTED ANSWER

        Re: Region crash when using multi-region environments. - Need any help...

        ‏2010-08-13T06:30:46Z  in response to eyley
        Ok... this confirms my understanding. This looks to be a known issue and has been fixed in 7.1.0.2 level. You can find the fix details here:

        http://www-933.ibm.com/support/fixcentral/swg/quickorder?parent=ibm~WebSphere&product=ibm/WebSphere/TXSeries+for+Multiplatforms&release=7.1.0.2&platform=AIX&function=fixId&fixids=7.1.0.2-TXSeries-AIX-InterimServiceFix2&includeRequisites=1&downloadMethod=http&source=fc

        TXSeries 7.1.0.2 InterimServiceFix 2

        This patch contains fixes for the following :
        APAR No Defect Abstract


        IZ76269 210342 PUT STATEMENT ON PL/I RESULTS IN ERZ027001E .
        IZ80091 210490 PL/I program doesn't log ProgaName to CPLI.out during exception
        IZ81023 210524 U5701 abend caused due to duplicate Application server Id
        IZ81026 210527 CICS-PL/I generates invalid code for EXEC CICS statements

        (C) Copyright International Business Machines Corporation, 2005. All Rights Reserved.
        <<End_of_APAR_Details>>
        HTH.

        Cheers,
        Hari
        • eyley
          eyley
          6 Posts
          ACCEPTED ANSWER

          Re: Region crash when using multi-region environments. - Need any help...

          ‏2010-08-16T00:53:23Z  in response to SystemAdmin
          Thank you Hari,

          I've received the interim fix and monitor it.

          Great Thanks to you...
  • SystemAdmin
    SystemAdmin
    308 Posts
    ACCEPTED ANSWER

    Re: Region crash when using multi-region environments. - Need any help...

    ‏2010-11-04T01:53:47Z  in response to eyley
    Hari77 wrote:
    Unfortunately the error information you have pasted in the post is too little to understand the problem. Would you be able to attach the console.<nnnnnn> and symrecs.<nnnnnn> files in this forum?. In general the logs are used by the application servers to store the transactional records and these logs are managed by the LogDaemon. There would be multiple logs (which you would find under the region's log directory) as there will be multiple application servers. Briefly, from the error it does look like that the application server is not able to open the log file, as it is already in use. Usually the logs are identified by the Application server ID (which starts from 101, 102, 103, ... and so on). So one way to check is to see in the console log if there are multiple application servers started with the same Application Server Id (a situation that shouldn't occur actually!!!)...then this can result in this error.

    In any case if you attach the console log, I can take a quick look.

    Cheers,
    Hari</nnnnnn></nnnnnn>

    Many thanks to your description! This is what I'm looking for.