Topic
  • 7 replies
  • Latest Post - ‏2014-06-26T13:29:40Z by XYKY_Hardik_Pathak
Yugan3R
Yugan3R
3 Posts

Pinned topic WebSeal error code: 0x38cf04d7 --- Third Party Server Not Responding...

‏2013-03-27T19:16:40Z |
Hi,

Encountering the below error, very frequently in the env, All the WebSeal Servers are Up and running.. and the junctions are active ... Didn't find any logs too.

DPWWA1239E
A third-party server is not responding. Possible causes: the server is down, there is a hung application on the server, or network problems. This is not a problem with the WebSEAL server.
Explanation:

A junctioned server is not responding to requests. Possible causes: junctioned server down, network problems, hung application on junctioned server.

Number:

0x38cf04d7
Please suggest any ideas to identify the issue...
Thanks
Updated on 2013-04-03T17:09:32Z at 2013-04-03T17:09:32Z by Yugan3R
  • hans.vandeweghe@be.ibm.com
    29 Posts

    Re: WebSeal error code: 0x38cf04d7 --- Third Party Server Not Responding...

    ‏2013-04-02T08:03:42Z  
    Hello,

    The approach that generally works for me when troubleshooting this sort of issue, is to start of with collecting the data described here:
    http://www-01.ibm.com/support/docview.wss?uid=swg27013231

    So the important bits are:

    • request.log
    • pdweb.snoop trace
    • pdweb.debug trace
    • TCP/IP trace

    You can look in the request.log to find entries which you suspect have returned the 500 DPWWA1239E error to the browser. Next you find these timestaps in the pdweb.debug trace, where you will also see the 500 error being returned to the browser. In the pdweb.debug trace you will find a thread(xxx) ID, which you can then track back in the pdweb.snoop trace.
    In the pdweb.snoop trace you can read information like IP address & Ports used. With this information you can then go to the TCP/IP trace, which is where you can understand really from the network layer what is going on when contacting the junctioned server.

    Alternatively you could skip the request.log & pdweb.debug stuff and just look in the pdweb.snoop or TCP/IP HTTP data for the DPWWA1239E string.

    One more thing to be aware of when using TAM 6.1.1 or ISAM 7.0.0 is the customized junction ping, which can also cause WebSEAL to serve the "third-party server not responding" error when a certain response is received from the back-end server. (but again you'd need something like pdweb.debug or pdweb.snoop to confirm)

    http://pic.dhe.ibm.com/infocenter/tivihelp/v2r1/topic/com.ibm.isam.doc_80/ameb_webseal_guide/reference/ref_resp_code_rule.html

    Good luck!

    Sincerely,
    Hans Vandeweghe

    BTW, feel free to raise a PMR with IBM Support if you want us to have a look at the MustGather data.
  • SystemAdmin
    SystemAdmin
    9855 Posts

    Re: WebSeal error code: 0x38cf04d7 --- Third Party Server Not Responding...

    ‏2013-04-02T10:17:58Z  
    Check the number of active webseal threads at the time of error using following command:

    s t <webseal instance> stats get pdweb.threads

    We faced a similar issue and found out webseal was running out of worker threads. If this is the case try increasing the number of webseal worker-threads.
  • Yugan3R
    Yugan3R
    3 Posts

    Re: WebSeal error code: 0x38cf04d7 --- Third Party Server Not Responding...

    ‏2013-04-03T17:02:58Z  
    Hello,

    The approach that generally works for me when troubleshooting this sort of issue, is to start of with collecting the data described here:
    http://www-01.ibm.com/support/docview.wss?uid=swg27013231

    So the important bits are:

    • request.log
    • pdweb.snoop trace
    • pdweb.debug trace
    • TCP/IP trace

    You can look in the request.log to find entries which you suspect have returned the 500 DPWWA1239E error to the browser. Next you find these timestaps in the pdweb.debug trace, where you will also see the 500 error being returned to the browser. In the pdweb.debug trace you will find a thread(xxx) ID, which you can then track back in the pdweb.snoop trace.
    In the pdweb.snoop trace you can read information like IP address & Ports used. With this information you can then go to the TCP/IP trace, which is where you can understand really from the network layer what is going on when contacting the junctioned server.

    Alternatively you could skip the request.log & pdweb.debug stuff and just look in the pdweb.snoop or TCP/IP HTTP data for the DPWWA1239E string.

    One more thing to be aware of when using TAM 6.1.1 or ISAM 7.0.0 is the customized junction ping, which can also cause WebSEAL to serve the "third-party server not responding" error when a certain response is received from the back-end server. (but again you'd need something like pdweb.debug or pdweb.snoop to confirm)

    http://pic.dhe.ibm.com/infocenter/tivihelp/v2r1/topic/com.ibm.isam.doc_80/ameb_webseal_guide/reference/ref_resp_code_rule.html

    Good luck!

    Sincerely,
    Hans Vandeweghe

    BTW, feel free to raise a PMR with IBM Support if you want us to have a look at the MustGather data.
    Hi Hans,

    Thanks for the reply and the provided steps to capture the required logs. PMR with Sev-1 has been opened and provided the logs as needed. Its clearly not a Web Seal issue but unable to identify the root cause that leading to this issue.
    The important observation we have captured is that, junction is in "not running " at the time of error .
    Clearly juction is unable to reach the end application. In webseal request logs , can only see 500 error at that time stamp .

    Thanks
  • Yugan3R
    Yugan3R
    3 Posts

    Re: WebSeal error code: 0x38cf04d7 --- Third Party Server Not Responding...

    ‏2013-04-03T17:09:32Z  
    Check the number of active webseal threads at the time of error using following command:

    s t <webseal instance> stats get pdweb.threads

    We faced a similar issue and found out webseal was running out of worker threads. If this is the case try increasing the number of webseal worker-threads.
    Hi Varun,

    THanks for the reply and sharing the ideas to identify the issue. I have tested the way you suggested, seems everything perfect with threads. There are only 3 active threads out of 100.
    Thanks
  • aanoufal
    aanoufal
    10 Posts

    Re: WebSeal error code: 0x38cf04d7 --- Third Party Server Not Responding...

    ‏2013-04-22T16:05:07Z  
    • Yugan3R
    • ‏2013-04-03T17:09:32Z
    Hi Varun,

    THanks for the reply and sharing the ideas to identify the issue. I have tested the way you suggested, seems everything perfect with threads. There are only 3 active threads out of 100.
    Thanks

    Hi,

    Any luck here. I am facing the same issue and still not able to identify the root cause. Users are able to access the application intermittently.

  • Mathanleo
    Mathanleo
    6 Posts

    Re: WebSeal error code: 0x38cf04d7 --- Third Party Server Not Responding...

    ‏2013-09-17T05:06:17Z  

    Hi Yugan3R,

    1.Possible cause may be Network connectivity issue (Connection timed out).

    by default WebSEAL server performs a periodic background 'ping' of each junctioned  Web server, to determine whether it is running.

    Make sure junctioned resource respond to ping request & it is not blocked in the firewall.

    Thanks,

    Mathan

    Updated on 2013-09-17T05:11:44Z at 2013-09-17T05:11:44Z by Mathanleo
  • XYKY_Hardik_Pathak
    XYKY_Hardik_Pathak
    2 Posts

    Re: WebSeal error code: 0x38cf04d7 --- Third Party Server Not Responding...

    ‏2014-06-26T13:29:40Z  

    Hi,

    This could be network issue, as we had a running setup with junctions created. And suddenly connectivity broke.

    On debugging, found the cross domain website where junction was pointing to had a new webserver. And I could not "wget" or telnet on that website from my webseal machine.

    Then we found that because they have a different machine, the IP was blocked by firewall and hence it was needed to be modified.

    Hope this helps someone who is struggling and find this suddenly into their working setup.

    Regards,

    Hardik