Topic
  • 25 replies
  • Latest Post - ‏2011-05-16T15:45:23Z by dhenderson
MaheshKharvi
MaheshKharvi
43 Posts

Pinned topic Errors seen in systemout.log during federation

‏2010-09-03T12:05:05Z |
When I tried to federate node to dmgr, lot of errors are shown in the SystemOut.log of dmgr.

I am attaching the log file.

here is the command

/sesitsbos8/was/INSTANCE1/profiles/base/bin/addNode.sh sesitsbos8 15113 -includeapps -startingport 15060
Updated on 2011-05-16T15:45:23Z at 2011-05-16T15:45:23Z by dhenderson
  • MaheshKharvi
    MaheshKharvi
    43 Posts

    Re: Errors seen in systemout.log during federation

    ‏2010-09-03T12:13:01Z  
    I am testing Beta on Solaris and Linux now and seen on both OS.
  • SystemAdmin
    SystemAdmin
    462 Posts

    Re: Errors seen in systemout.log during federation

    ‏2010-09-03T12:20:21Z  
    Hi,

    Are you trying to add node INSTANCE2 from INSTANCE1 as it seems that there is a problem in the node name you are trying to add according to this error of no such file for INATANCE2?

    Caused by: org.apache.ws.commons.schema.XmlSchemaException: /sesitsbos8/was/INSTANCE2/profiles/dmgr/WEB-INF/classes/wsdl/J2EEManagedObject.xsd (No such file or directory)
  • MaheshKharvi
    MaheshKharvi
    43 Posts

    Re: Errors seen in systemout.log during federation

    ‏2010-09-03T12:23:43Z  
    Hi,

    Are you trying to add node INSTANCE2 from INSTANCE1 as it seems that there is a problem in the node name you are trying to add according to this error of no such file for INATANCE2?

    Caused by: org.apache.ws.commons.schema.XmlSchemaException: /sesitsbos8/was/INSTANCE2/profiles/dmgr/WEB-INF/classes/wsdl/J2EEManagedObject.xsd (No such file or directory)
    Oops sorry, I attached INSTANCE2 log file, but posted command used for INSTANCE1.

    Both profiles are in the same instance and I created two instances to make sure the error exists.
  • SystemAdmin
    SystemAdmin
    462 Posts

    Re: Errors seen in systemout.log during federation

    ‏2010-09-03T15:43:07Z  
    Oops sorry, I attached INSTANCE2 log file, but posted command used for INSTANCE1.

    Both profiles are in the same instance and I created two instances to make sure the error exists.
    Did you try using admin console and got same error?
  • MaheshKharvi
    MaheshKharvi
    43 Posts

    Re: Errors seen in systemout.log during federation

    ‏2010-09-06T07:15:05Z  
    Did you try using admin console and got same error?
    Hi AMonsef,

    I didn't see error in SystemOut.log this time when used admin console.
    But "ADMU0040E: Exception on MBean invocation " still exists.

    I am attaching the addnode output.
  • MaheshKharvi
    MaheshKharvi
    43 Posts

    Re: Errors seen in systemout.log during federation

    ‏2010-09-06T10:04:29Z  
    Hi AMonsef,

    I didn't see error in SystemOut.log this time when used admin console.
    But "ADMU0040E: Exception on MBean invocation " still exists.

    I am attaching the addnode output.
    Found that plugin path in WebServer definition was incorrect, updated that before the federation and no errors now.
  • MaheshKharvi
    MaheshKharvi
    43 Posts

    Re: Errors seen in systemout.log during federation

    ‏2010-09-06T10:52:19Z  
    Found that plugin path in WebServer definition was incorrect, updated that before the federation and no errors now.
    The error still exists in SystemOut.log though no error in addNode.log file.

    Caused by: java.lang.RuntimeException: org.apache.ws.commons.schema.XmlSchemaException: /sesitsbos8/was/INSTANCE1/profiles/dmg
    r/WEB-INF/classes/wsdl/WS-MetadataExchange-2004_09.xsd (No such file or directory)
  • MaheshKharvi
    MaheshKharvi
    43 Posts

    Re: Errors seen in systemout.log during federation

    ‏2010-09-06T10:52:38Z  
    .
  • SystemAdmin
    SystemAdmin
    462 Posts

    Re: Errors seen in systemout.log during federation

    ‏2010-09-06T12:35:46Z  
    The error still exists in SystemOut.log though no error in addNode.log file.

    Caused by: java.lang.RuntimeException: org.apache.ws.commons.schema.XmlSchemaException: /sesitsbos8/was/INSTANCE1/profiles/dmg
    r/WEB-INF/classes/wsdl/WS-MetadataExchange-2004_09.xsd (No such file or directory)
    Hi,

    Do you mean that node addedd successfully using admin console after correcting plugiin path and that error still shown in systemOut.log!!

    Regards,
    Ahmed
  • MaheshKharvi
    MaheshKharvi
    43 Posts

    Re: Errors seen in systemout.log during federation

    ‏2010-09-06T12:56:55Z  
    Hi,

    Do you mean that node addedd successfully using admin console after correcting plugiin path and that error still shown in systemOut.log!!

    Regards,
    Ahmed
    Hi,

    Federation is successful in all scenarios, but errors are shown in the logfiles.

    After fixing the plugin path, no error found in addNode.log file.

    But still SystemOut.log was showing "profiles/dmgr2/WEB-INF/classes/wsdl/WS-MetadataExchange-2004_09.xsd (No such file or directory)" and many more.

    I found significant files are not present under DMR_PROFILE_CONFIGROOT/nd_cellname/applications/WebSphereWSDM.ear/deployments/WebSphereWSDM/WebSphereWSDM.war/WEB-INF/classes/wsdl compared to WAS_HOME/systemApps//WebSphereWSDM.ear/WebSphereWSDM.war/WEB-INF/classes/wsdl.

    Now I copied systemApps wsdl directory dmgr_profile applications directory prior to federation and I dont see any error message now.

    IBM has to look into it why not all files are getting copied to applications directory.
  • SystemAdmin
    SystemAdmin
    462 Posts

    Re: Errors seen in systemout.log during federation

    ‏2010-09-06T13:09:25Z  
    Hi,

    Federation is successful in all scenarios, but errors are shown in the logfiles.

    After fixing the plugin path, no error found in addNode.log file.

    But still SystemOut.log was showing "profiles/dmgr2/WEB-INF/classes/wsdl/WS-MetadataExchange-2004_09.xsd (No such file or directory)" and many more.

    I found significant files are not present under DMR_PROFILE_CONFIGROOT/nd_cellname/applications/WebSphereWSDM.ear/deployments/WebSphereWSDM/WebSphereWSDM.war/WEB-INF/classes/wsdl compared to WAS_HOME/systemApps//WebSphereWSDM.ear/WebSphereWSDM.war/WEB-INF/classes/wsdl.

    Now I copied systemApps wsdl directory dmgr_profile applications directory prior to federation and I dont see any error message now.

    IBM has to look into it why not all files are getting copied to applications directory.
    Hi,

    Good news!

    Can you please tell me the components of your application federated with that node trying to make same scenario on my side checking if it will happen or not?

    Regards,
    Ahmed
  • MaheshKharvi
    MaheshKharvi
    43 Posts

    Re: Errors seen in systemout.log during federation

    ‏2010-09-06T13:12:46Z  
    Hi,

    Good news!

    Can you please tell me the components of your application federated with that node trying to make same scenario on my side checking if it will happen or not?

    Regards,
    Ahmed
    ND, IHS and PLUGIN installation
    Create SA profile (without IHSnode) and DMGR profile
    Start DMGR profile
    Run addnode command with -includeapps.
  • SystemAdmin
    SystemAdmin
    462 Posts

    Re: Errors seen in systemout.log during federation

    ‏2010-09-06T13:20:00Z  
    ND, IHS and PLUGIN installation
    Create SA profile (without IHSnode) and DMGR profile
    Start DMGR profile
    Run addnode command with -includeapps.
    Thanks but I mean the enterprise application componenets you included in your federation (missing files found on tham after federation) or you just include default apps on that node?
  • MaheshKharvi
    MaheshKharvi
    43 Posts

    Re: Errors seen in systemout.log during federation

    ‏2010-09-06T13:21:52Z  
    Thanks but I mean the enterprise application componenets you included in your federation (missing files found on tham after federation) or you just include default apps on that node?
    WebSphereWSDM is a systemApp for which it is trying to find the files.
    I didn't install any new applications, all are default.
  • SystemAdmin
    SystemAdmin
    462 Posts

    Re: Errors seen in systemout.log during federation

    ‏2010-09-06T15:35:44Z  
    WebSphereWSDM is a systemApp for which it is trying to find the files.
    I didn't install any new applications, all are default.
    OK, thank you, i will update you!
  • SystemAdmin
    SystemAdmin
    462 Posts

    Re: Errors seen in systemout.log during federation

    ‏2010-09-08T13:20:56Z  
    OK, thank you, i will update you!
    Hi,

    I tried to add node including apps smoothly with no errors, so can you try to remove that node then adding it again or creating another profile and try to federate that node to ND?

    Regards,
    Ahmed
  • BarryF
    BarryF
    19 Posts

    Re: Errors seen in systemout.log during federation

    ‏2010-09-10T15:07:29Z  
    Hi,

    I tried to add node including apps smoothly with no errors, so can you try to remove that node then adding it again or creating another profile and try to federate that node to ND?

    Regards,
    Ahmed
    Hello Mahesh,

    Sorry for the delay in responding to this post. Thanks to Ahmed for your participation in this thread. The WAS development team would like to investigate the issues you have encountered to ensure any potential defects are addressed in future updates to our product. I had asked several of our testers to attempt to recreate the problem(s) you encountered in our lab but I see that you have updated the thread as 'Answered'. Do you feel there are outstanding issues with the Beta that we should address in future offerings?

    Regards,
    Barry Foster
    WebSphere Application Server Development Manager
  • SystemAdmin
    SystemAdmin
    462 Posts

    Re: Errors seen in systemout.log during federation

    ‏2010-09-10T22:23:39Z  
    • BarryF
    • ‏2010-09-10T15:07:29Z
    Hello Mahesh,

    Sorry for the delay in responding to this post. Thanks to Ahmed for your participation in this thread. The WAS development team would like to investigate the issues you have encountered to ensure any potential defects are addressed in future updates to our product. I had asked several of our testers to attempt to recreate the problem(s) you encountered in our lab but I see that you have updated the thread as 'Answered'. Do you feel there are outstanding issues with the Beta that we should address in future offerings?

    Regards,
    Barry Foster
    WebSphere Application Server Development Manager
    Mahesh,

    Thank you for reporting this. We are trying to recreate this problem in the lab here at IBM in case it can be resolved in a future update. So far, addNode has been successful in all of our tests on the beta. If this is still a concern, do you mind telling us a little more about your setup so we can determine what we are doing differently? Our environment is one Linux machine with WAS8 beta ND (x86_64), IHS, and web server plugin. There are two profiles created through the manageprofiles command like so:
    manageprofiles -create -templatePath <WAS_HOME>/profileTemplates/dmgr
    manageprofiles -create -templatePath <WAS_HOME>/profileTemplates/default

    After starting the dmgr, addNode is run as follows from WAS_HOME:
    addNode <host> <dmgr_port> -includeapps -startingport 15060 -profileName AppSrv01

    Is there any way in which this environment and these steps differ from yours?

    Thanks,

    Travis Rinn
    WebSphere Application Server Persona Test
  • MaheshKharvi
    MaheshKharvi
    43 Posts

    Re: Errors seen in systemout.log during federation

    ‏2010-09-13T08:04:15Z  
    Mahesh,

    Thank you for reporting this. We are trying to recreate this problem in the lab here at IBM in case it can be resolved in a future update. So far, addNode has been successful in all of our tests on the beta. If this is still a concern, do you mind telling us a little more about your setup so we can determine what we are doing differently? Our environment is one Linux machine with WAS8 beta ND (x86_64), IHS, and web server plugin. There are two profiles created through the manageprofiles command like so:
    manageprofiles -create -templatePath <WAS_HOME>/profileTemplates/dmgr
    manageprofiles -create -templatePath <WAS_HOME>/profileTemplates/default

    After starting the dmgr, addNode is run as follows from WAS_HOME:
    addNode <host> <dmgr_port> -includeapps -startingport 15060 -profileName AppSrv01

    Is there any way in which this environment and these steps differ from yours?

    Thanks,

    Travis Rinn
    WebSphere Application Server Persona Test
    Travis,

    Thank you for working on this.
    Still I am facing the same problem.
    I am not seeing any error during the federation, the output indicates successfully federated, when refered dmgr logs, it shows some of the files are missing for WebSphereWSDM.

    I tried different configuration,
    Installed only WAS ND, created two profiles with default and dmgr as the templates without any other options to manageprofile.
    Ran the addnode command, still errors are shown in the dmgr systemOut.log.
  • RohitK
    RohitK
    38 Posts

    Re: Errors seen in systemout.log during federation

    ‏2010-09-13T22:16:45Z  
    Travis,

    Thank you for working on this.
    Still I am facing the same problem.
    I am not seeing any error during the federation, the output indicates successfully federated, when refered dmgr logs, it shows some of the files are missing for WebSphereWSDM.

    I tried different configuration,
    Installed only WAS ND, created two profiles with default and dmgr as the templates without any other options to manageprofile.
    Ran the addnode command, still errors are shown in the dmgr systemOut.log.
    Mahesh,

    We have a process in place to diagnose and debug node federation issues. If you are amenable to repeating the scenario, can you please follow the steps listed here in the MustGather document for Federation or removal of node issues
    http://www-01.ibm.com/support/docview.wss?rs=180&context=SSCMPDZ&q1=MustGatherDocument&uid=swg21196227&loc=en_US&cs=utf-8&lang=en

    The output of the collector script in <WAS_HOME>/bin directory will also help us in debugging this issue. Please note that the Collector tool (collector.bat/sh) can not be run in the WebSphere install directory or a WebSphere install sub directory.
    --Thanks,
    Rohit Kelapure
  • MaheshKharvi
    MaheshKharvi
    43 Posts

    Re: Errors seen in systemout.log during federation

    ‏2010-09-14T06:01:18Z  
    • RohitK
    • ‏2010-09-13T22:16:45Z
    Mahesh,

    We have a process in place to diagnose and debug node federation issues. If you are amenable to repeating the scenario, can you please follow the steps listed here in the MustGather document for Federation or removal of node issues
    http://www-01.ibm.com/support/docview.wss?rs=180&context=SSCMPDZ&q1=MustGatherDocument&uid=swg21196227&loc=en_US&cs=utf-8&lang=en

    The output of the collector script in <WAS_HOME>/bin directory will also help us in debugging this issue. Please note that the Collector tool (collector.bat/sh) can not be run in the WebSphere install directory or a WebSphere install sub directory.
    --Thanks,
    Rohit Kelapure
    I never said federation is failure in my case, It was successful from very begining, but I see error messages in dmgr SystemOut.log.

    This is the only concern from my end, I am able to perform all tasks after with federeted node.
  • MaheshKharvi
    MaheshKharvi
    43 Posts

    Re: Errors seen in systemout.log during federation

    ‏2010-09-17T12:35:58Z  
    Travis,

    Can I have some update on this.

    Thanks,
    Mahesh
  • SystemAdmin
    SystemAdmin
    462 Posts

    Re: Errors seen in systemout.log during federation

    ‏2010-09-17T14:29:03Z  
    Travis,

    Can I have some update on this.

    Thanks,
    Mahesh
    Mahesh,

    Thanks for the additional information on environment. You had mentioned Solaris, and we are also trying to reproduce on Solaris now since the problem has not been found yet on Linux. Knowing that it can happen without supporting products installed helps. There must still be some key difference between our lab environment and yours that explains why we are not hitting the exception regularly, and we are looking for that difference on our side. If you have the time to answer any of the following questions, it would help in providing a faster resolution, but we will continue to work on this either way.

    1. Have you tried this at any time on any platform, looked in the logs for this exception, and not seen it? If so, what platform was that on? Since there are some differences between platforms, having an idea if it is all of them or some of them would make a difference.
    2. Do you know of any processes that are running on both your Linux and Solaris machine that could be interfering? That may be why we have not been able to reproduce in the IBM lab so far - perhaps you have some additional software running on your systems beyond WAS and the default configuration, i.e. a database or software required by your company.
    3. Is there anything in the logs/ffdc directories for either profile? Sometimes the exceptions here provide us with better detail of a failure.

    Thanks,

    Travis Rinn
    WebSphere Application Server Persona Test
  • SystemAdmin
    SystemAdmin
    462 Posts

    Re: Errors seen in systemout.log during federation

    ‏2010-09-17T16:02:53Z  
    Mahesh,

    Thanks for the additional information on environment. You had mentioned Solaris, and we are also trying to reproduce on Solaris now since the problem has not been found yet on Linux. Knowing that it can happen without supporting products installed helps. There must still be some key difference between our lab environment and yours that explains why we are not hitting the exception regularly, and we are looking for that difference on our side. If you have the time to answer any of the following questions, it would help in providing a faster resolution, but we will continue to work on this either way.

    1. Have you tried this at any time on any platform, looked in the logs for this exception, and not seen it? If so, what platform was that on? Since there are some differences between platforms, having an idea if it is all of them or some of them would make a difference.
    2. Do you know of any processes that are running on both your Linux and Solaris machine that could be interfering? That may be why we have not been able to reproduce in the IBM lab so far - perhaps you have some additional software running on your systems beyond WAS and the default configuration, i.e. a database or software required by your company.
    3. Is there anything in the logs/ffdc directories for either profile? Sometimes the exceptions here provide us with better detail of a failure.

    Thanks,

    Travis Rinn
    WebSphere Application Server Persona Test
    Mahesh,

    Don't worry about the above questions anymore. This has now been reproduced in the lab on Solaris. We have enough information for the time being. Once again, we appreciate your help in bringing this to our attention so that we can fix the problem in the future.

    Thanks,

    Travis Rinn
    WebSphere Application Server Persona Test