Topic
  • 5 replies
  • Latest Post - ‏2015-07-09T23:44:02Z by bpaskin
wspherian
wspherian
4 Posts

Pinned topic deployment manager's cosole slowness

‏2013-04-29T12:44:23Z |

Hello

We have a websphere application server cluster 8.0.0.5, and we use ldap as user account repository, the problem is that deployment manager's console is vary slow and login to console takes a  lot of  time.

We see the following lines in deployment manager's systemOut.log:

[4/29/13 8:48:49:584 ] 00000039 LdapRegistryI A   SECJ0419I: The user registry is currently connected to the LDAP server ldap://ldap:636.
[4/29/13 9:01:41:370 ] 00000003 ThreadMonitor W   WSVR0605W: Thread "WebContainer : 4" (00000032) has been active for 757032 milliseconds and may be hung.  There is/are 1 thread(s) in total in the server that may be hung.
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(SocketInputStream.java:140)
    at com.ibm.jsse2.b.a(b.java:111)
    at com.ibm.jsse2.b.a(b.java:90)
    at com.ibm.jsse2.SSLSocketImpl.a(SSLSocketImpl.java:384)
    at com.ibm.jsse2.SSLSocketImpl.a(SSLSocketImpl.java:91)
    at com.ibm.jsse2.f.read(f.java:43)
    at java.io.BufferedInputStream.fill(BufferedInputStream.java:229)
    at java.io.BufferedInputStream.read(BufferedInputStream.java:248)
    at org.apache.soap.util.net.HTTPUtils.post(HTTPUtils.java:590)
    at org.apache.soap.transport.http.SOAPHTTPConnection.send(SOAPHTTPConnection.java:425)
    at org.apache.soap.rpc.Call.WASinvoke(Call.java:451)
    at com.ibm.ws.management.connector.soap.SOAPConnectorClient$8.run(SOAPConnectorClient.java:844)
    at com.ibm.ws.security.util.AccessController.doPrivileged(AccessController.java:118)
    at com.ibm.ws.management.connector.soap.SOAPConnectorClient.invokeTemplateOnce(SOAPConnectorClient.java:837)
    at com.ibm.ws.management.connector.soap.SOAPConnectorClient.invokeTemplate(SOAPConnectorClient.java:682)
    at com.ibm.ws.management.connector.soap.SOAPConnectorClient.invokeTemplate(SOAPConnectorClient.java:672)
    at com.ibm.ws.management.connector.soap.SOAPConnectorClient.queryNames(SOAPConnectorClient.java:584)
    at com.ibm.ws.management.connector.soap.SOAPConnectorClient.invoke(SOAPConnectorClient.java:509)
    at $Proxy39.queryNames(Unknown Source)
    at com.ibm.ws.management.AdminClientImpl.queryNames(AdminClientImpl.java:108)
    at com.ibm.ws.management.AdminServiceImpl.queryNames(AdminServiceImpl.java:664)
    at com.ibm.ws.console.tpv.server.ServerStateManager.detectServerStates(ServerStateManager.java:221)
    at com.ibm.ws.console.tpv.server.ServerStateManager.getState(ServerStateManager.java:105)
    at com.ibm.ws.console.tpv.controller.ServerListController.setupCollectionForm(ServerListController.java:162)
    at com.ibm.ws.console.tpv.controller.ServerListController.execute(ServerListController.java:107)
    at com.ibm.ws.console.tpv.controller.ServerListController.perform(ServerListController.java:68)
    at org.apache.struts.tiles.TilesRequestProcessor.processTilesDefinition(Unknown Source)
    at org.apache.struts.tiles.TilesRequestProcessor.processForwardConfig(Unknown Source)
    at org.apache.struts.action.RequestProcessor.process(Unknown Source)
    at org.apache.struts.action.ActionServlet.process(Unknown Source)
    at org.apache.struts.action.ActionServlet.doGet(Unknown Source)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:575)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
    at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1229)
    at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:774)
    at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:456)
    at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.handleRequest(ServletWrapperImpl.java:178)
    at com.ibm.ws.webcontainer.filter.WebAppFilterChain.invokeTarget(WebAppFilterChain.java:136)
    at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:97)
    at com.ibm.ws.console.core.servlet.WSCUrlFilter.setUpCommandAssistance(WSCUrlFilter.java:950)
    at com.ibm.ws.console.core.servlet.WSCUrlFilter.continueStoringTaskState(WSCUrlFilter.java:499)
    at com.ibm.ws.console.core.servlet.WSCUrlFilter.doFilter(WSCUrlFilter.java:320)
    at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:195)
    at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:91)
    at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:928)
    at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1025)
    at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:87)
    at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:895)
    at com.ibm.ws.webcontainer.WSWebContainer.handleRequest(WSWebContainer.java:1662)
    at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:195)
    at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:452)
    at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewRequest(HttpInboundLink.java:511)
    at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.processRequest(HttpInboundLink.java:305)
    at com.ibm.ws.http.channel.inbound.impl.HttpICLReadCallback.complete(HttpICLReadCallback.java:83)
    at com.ibm.ws.ssl.channel.impl.SSLReadServiceContext$SSLReadCompletedCallback.complete(SSLReadServiceContext.java:1784)
    at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:165)
    at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
    at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
    at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138)
    at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204)
    at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775)
    at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
    at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1690)

 

Best Regards,

  • markernst
    markernst
    1 Post

    Re: deployment manager's cosole slowness

    ‏2013-04-29T19:40:01Z  

    LDAP connections won't be using the Web Container, so I don't think they are related.  (LDAP from the DMgr connects directly to the LDAP server)  you can turn up the logging on anything in WebSphere to find out the specifics of what is happening.  Just zoom in on the part you are interested in.

    Or you can try some basic steps to see if it clears up.  Does a restart of DMgr help?  Do you logout of Dmgr everytime or do you leave lots of ../wstemp/ files around?  Do you do many deployments?  All of that leaves temporary files that will slow down the DMgr.  (clean them up!) Do you have enough CPU and RAM allocated on the machine?  is the JVM for Dmgr allocated enough RAM?  That is changeable/ tunable.  Is the LDAP server slow to respond to everything?  Use LDAP browser or other client to test.

  • gas
    gas
    929 Posts

    Re: deployment manager's cosole slowness

    ‏2013-04-30T06:40:40Z  

    Hi,

    Looks like Dmgr is trying to get server states:

    ServerStateManager.getState()

    check if you have connection between dmgr and node agent from both sides (via ping and telnet on soap port). Maybe something is blocking communication.

    Also check nodeagent status in the console.

    Gas

  • wspherian
    wspherian
    4 Posts

    Re: deployment manager's cosole slowness

    ‏2013-05-08T06:20:15Z  
    • gas
    • ‏2013-04-30T06:40:40Z

    Hi,

    Looks like Dmgr is trying to get server states:

    ServerStateManager.getState()

    check if you have connection between dmgr and node agent from both sides (via ping and telnet on soap port). Maybe something is blocking communication.

    Also check nodeagent status in the console.

    Gas

    Hi, Thanks for your advices, we clean up temporary files and  increase deployment manager's heap size,  CPU usage in deployment manager's machine is less than 2%, and RAM usage is less than 1 GB and we have 8GB RAM. Now we can login to deployment manager's console more quicker than before but some problems still remain, when we want to connect to nodes for example when we want to see current activity of nodes (monitoring and tuning > performance viewer > current activity) it takes too long time and we saw in the log that some webcontainer's thread are hung (log is the same as previous post) we check connection between deployment manager's nodes and agents it seems that it is ok and both side can telnet each other on soap port.

    Thanks

  • CPDK
    CPDK
    1 Post

    Re: deployment manager's cosole slowness

    ‏2015-07-09T18:50:35Z  
    • wspherian
    • ‏2013-05-08T06:20:15Z

    Hi, Thanks for your advices, we clean up temporary files and  increase deployment manager's heap size,  CPU usage in deployment manager's machine is less than 2%, and RAM usage is less than 1 GB and we have 8GB RAM. Now we can login to deployment manager's console more quicker than before but some problems still remain, when we want to connect to nodes for example when we want to see current activity of nodes (monitoring and tuning > performance viewer > current activity) it takes too long time and we saw in the log that some webcontainer's thread are hung (log is the same as previous post) we check connection between deployment manager's nodes and agents it seems that it is ok and both side can telnet each other on soap port.

    Thanks

    Hi, I know it's been a while, but did you figure out why your login to the console was really slow? I am having the same issue on WAS 8.0.0.8 base edition. Thanks!

  • bpaskin
    bpaskin
    5251 Posts

    Re: deployment manager's cosole slowness

    ‏2015-07-09T23:44:02Z  
    • CPDK
    • ‏2015-07-09T18:50:35Z

    Hi, I know it's been a while, but did you figure out why your login to the console was really slow? I am having the same issue on WAS 8.0.0.8 base edition. Thanks!

    Hi, change the heap size on your Dmgr.

     

    Brian