IBM Support

IV10815: NCP_NCOGATE CORE AT CNCOCONNECTION::READERMAINLOOP: LOOP HAS BEEN TERMINATED

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as program error.

Error description

  • The ncp_ncogate process is coring.
    
    The stack trace shows the problem is in
    CNcoConnection::ReaderMainLoop(). As the name suggests, we
    should just keep looping round in that function. When the
    GetIDUCData() call times out, it returns false, we exit the loop
    and terminate. Pseudo-code is below
    
    ReaderMainLoop()
    while ( rc )
    {
         rc = DataProcess()->GetIDUCData()
    }
    CRivError( terminate and dump core )
    
    
    ---ncp_ncogate.ATFNETP.log   ----
    Warning: W-NCO-001-006: [13t] Successfully failed back to
    PRIMARY Object Server AGG_V
    Error: E-IPC-053-003: OpenClient (Layer = 1, Origin = 2,
    Severity = 2, Number = 63) - ct_results(): user api layer:
    internal Client Library error: Read from the server has timed
    out.
    Error: E-IPC-053-003: OpenClient (Layer = 1, Origin = 2,
    Severity = 2, Number = 63) - ct_results(): user api layer:
    internal Client Library error: Read from the server has timed
    out.
    Error: E-IPC-053-003: OpenClient (Layer = 1, Origin = 1,
    Severity = 1, Number = 50) - ct_cmd_drop(): user api layer:
    external error: The connection has been marked dead.
    Warning: W-RIV-002-128: [13t] CNcoConnection.cc(1052)
    CNcoConnection::DataProcess: Failed to fetch raw IDUC data from
    the server.
    Fatal: F-RIV-002-127: [13t] CNcoConnection.cc(728)
    CNcoConnection::ReaderMainLoop: loop has been terminated.
    Information: I-MOM-001-001: [1t] ncp_ncogate[27201] Version 3.8
    (Build 61) becoming Primary
    
    --- core pstack ----------------
    core 'core_10791/core' of 10791: ncp_ncogate -domain ATFNETP
    -server AGG_V -latency 100000 -debug 4 -me
    -----------------  lwp# 1 / thread# 1  --------------------
     7f44d8b8 __pollsys (ffbff040, 1, ffbff130, 0, 0, 0) + 8
     7f3e8ff0 pselect  (ffbff040, 7f4b4790, 7f4b4790, 40, ffbff130,
    0) + 1c8
     7f3e9368 select   (11, ffbff2a0, 0, 0, ffbff198, 0) + a0
     7f9198e8 _WaitForInput (ac4f0, ffbff390, 1, 1, 7f9165f4, 33cd8)
    + 114
     7f919b48 _rvevm_MainLoop (ac4f0, 1, 7f919af0, 2, 7f9bd1e0,
    fffca000) + 58
     7f916094 rv_MainLoop (ac930, 1, 116428, a4ff8, 1, 38088) + 28
     7f90a364 __1cJCRivRvNetHRRNDoIt6M_v_ (a4ff8, ae0d0, 0, 1, 1,
    a4ff8) + 78
     7f904340 __1cKCRivEngineGREDoIt6M_v_ (a4ff8, f, 7fa1c868,
    7fa19f58, 78, 1) + ac
     7f7c20b0 __1cPCRivApplicationHRAStart6M_v_ (85f10, b9568,
    28d44, 117e48, 5177c, 7f84be27) + 168
     0003132c __1cHRivMain6Fippc_i_ (4, 0, 5201c, a4ff8, fffca000,
    3c6b8) + 2b4
     00031050 main     (d, ffbff704, ffbff73c, 4fb38, fffe1540,
    1e800) + 40
     0001ed90 _start   (0, 0, 0, 0, 0, 0) + 108
    -----------------  lwp# 2 / thread# 2  --------------------
    

Local fix

  • A check on Obect Server performance should also be performed.
    

Problem summary

  • ****************************************************************
    * USERS AFFECTED:                                              *
    * Can occur when the Omnibus server is under sufficient load   *
    * to not reply to ncp_ncogate                                  *
    ****************************************************************
    * PROBLEM DESCRIPTION:                                         *
    * ncp_ncogate cores with this in the logs:                     *
    *                                                              *
    * Warning: W-NCO-001-006: [13t] Successfully failed back to    *
    * PRIMARY Object Server AGG_V                                  *
    * Error: E-IPC-053-003: OpenClient (Layer = 1, Origin = 2,     *
    * Severity = 2, Number = 63) - ct_results(): user api layer:   *
    * internal Client Library error: Read from the server has      *
    * timed                                                        *
    * out.                                                         *
    * Error: E-IPC-053-003: OpenClient (Layer = 1, Origin = 2,     *
    * Severity = 2, Number = 63) - ct_results(): user api layer:   *
    * internal Client Library error: Read from the server has      *
    * timed                                                        *
    * out.                                                         *
    * Error: E-IPC-053-003: OpenClient (Layer = 1, Origin = 1,     *
    * Severity = 1, Number = 50) - ct_cmd_drop(): user api layer:  *
    * external error: The connection has been marked dead.         *
    * Warning: W-RIV-002-128: [13t] CNcoConnection.cc(1052)        *
    * CNcoConnection::DataProcess: Failed to fetch raw IDUC data   *
    * from                                                         *
    * the server.                                                  *
    * Fatal: F-RIV-002-127: [13t] CNcoConnection.cc(728)           *
    * CNcoConnection::ReaderMainLoop: loop has been terminated.    *
    * Information: I-MOM-001-001: [1t] ncp_ncogate[27201] Version  *
    * 3.8                                                          *
    * (Build 61) becoming Primary                                  *
    ****************************************************************
    * RECOMMENDATION:                                              *
    * Install the IV10815 test fix or ITNM 3.8 FP7                 *
    * | fix pack | 3.9.0-ITNMIP-FP0001                             *
    * | fix pack | 3.8.0-ITNMIP-FP0007                             *
    ****************************************************************
    

Problem conclusion

  • | fix pack | 3.9.0-ITNMIP-FP0001
    | fix pack | 3.8.0-ITNMIP-FP0007
    

Temporary fix

Comments

APAR Information

  • APAR number

    IV10815

  • Reported component name

    NC/PRECISIONIP

  • Reported component ID

    5724O52RC

  • Reported release

    380

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt

  • Submitted date

    2011-11-16

  • Closed date

    2012-01-17

  • Last modified date

    2012-02-09

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    NC/PRECISIONIP

  • Fixed component ID

    5724O52RC

Applicable component levels

  • R330 PSN

       UP

  • R330 PSY

       UP

  • R340 PSN

       UP

  • R340 PSY

       UP

  • R350 PSN

       UP

  • R350 PSY

       UP

  • R360 PSN

       UP

  • R360 PSY

       UP

  • R370 PSN

       UP

  • R370 PSY

       UP

  • R380 PSN

       UP

  • R380 PSY

       UP

  • R390 PSN

       UP

  • R390 PSY

       UP

[{"Business Unit":{"code":"BU048","label":"IBM Software"},"Product":{"code":"SSCP984","label":"Discovery and RCA"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"380","Edition":"","Line of Business":{"code":"","label":""}}]

Document Information

Modified date:
09 February 2012