Topic
  • 18 replies
  • Latest Post - ‏2013-10-28T18:56:51Z by chuck1024
SystemAdmin
SystemAdmin
1485 Posts

Pinned topic Serializable Issue ?

‏2013-04-01T15:35:45Z |
Hi,

I am using IBM WXS 8.5.

After enabling the Extreme Scale caching for Session Management, I am now seeing the following errors/warnings in the SystemOut.log for Portal Server 8.

==
0000006a FfdcProvider W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on /opt/IBM/WebSphere/wp_profile/logs/ffdc/WebSphere_Portal_160ef41d_13.04.01_09.45.58.0298634360710023315755.txt com.ibm.ws.objectgrid.map.LogSequenceImpl.serialize 599
4/1/13 9:45:58:032 CDT 0000006a LogSequenceIm W CWOBJ0006W: An exception occurred: com.ibm.websphere.objectgrid.ObjectGridRuntimeException: java.io.NotSerializableException: com.ibm.wps.engine.PortalRequestWrapper
at com.ibm.ws.objectgrid.map.BaseMap.objectToBytes(BaseMap.java:11894)
at com.ibm.ws.objectgrid.DiffMapValue.getCurrentBytes(DiffMapValue.java:839)
at com.ibm.ws.objectgrid.map.LogSequenceImpl.serialize(LogSequenceImpl.java:1163)
at com.ibm.ws.objectgrid.map.LogSequenceImpl.serialize(LogSequenceImpl.java:839)
at com.ibm.websphere.objectgrid.plugins.LogSequenceTransformer.serialize(LogSequenceTransformer.java:100)
at com.ibm.ws.objectgrid.event.ClientReadWriteRequestSystemEvent.writeEvent(ClientReadWriteRequestSystemEvent.java:286)
at com.ibm.ws.objectgrid.client.ORBClientCoreMessageHandler.sendMessage(ORBClientCoreMessageHandler.java:328)
at com.ibm.ws.objectgrid.client.ORBClientCoreMessageHandler.sendReadWriteRequest(ORBClientCoreMessageHandler.java:1044)
at com.ibm.ws.objectgrid.client.RemoteTransactionCallbackImpl.processReadWriteRequestAndResponse(RemoteTransactionCallbackImpl.java:970)
at com.ibm.ws.objectgrid.client.RemoteTransactionCallbackImpl.commit(RemoteTransactionCallbackImpl.java:283)
at com.ibm.ws.objectgrid.SessionImpl.commit(SessionImpl.java:1848)
at com.ibm.ws.xs.sessionmanager.SessionReplicationManager.run(SessionReplicationManager.java:719)
at java.util.Timer$TimerImpl.run(Timer.java:296)
Caused by: java.io.NotSerializableException: com.ibm.wps.engine.PortalRequestWrapper
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1538)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1503)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1420)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1538)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1503)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1420)

==========httprequest:::::Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR
4/1/13 9:46:47:171 CDT 00000072 ObjectGridImp I CWOBJ4700I: The map name objectgridSessionAttributeEvicted matched the regular expression of template map objectgridSessionAttribute.*. The objectgridSessionAttributeEvicted map has been created for ObjectGrid PORTAL8SHARED_SESSIONS.
4/1/13 9:46:47:703 CDT 0000006a LogSequenceIm W CWOBJ0006W: An exception occurred: com.ibm.websphere.objectgrid.ObjectGridRuntimeException: java.io.NotSerializableException: com.ibm.wps.engine.PortalRequestWrapper
at com.ibm.ws.objectgrid.map.BaseMap.objectToBytes(BaseMap.java:11894)
at com.ibm.ws.objectgrid.DiffMapValue.getCurrentBytes(DiffMapValue.java:839)
at com.ibm.ws.objectgrid.map.LogSequenceImpl.serialize(LogSequenceImpl.java:1163)
at com.ibm.ws.objectgrid.map.LogSequenceImpl.serialize(LogSequenceImpl.java:839)
at com.ibm.websphere.objectgrid.plugins.LogSequenceTransformer.serialize(LogSequenceTransformer.java:100)
at com.ibm.ws.objectgrid.event.ClientReadWriteRequestSystemEvent.writeEvent(ClientReadWriteRequestSystemEvent.java:286)
at com.ibm.ws.objectgrid.client.ORBClientCoreMessageHandler.sendMessage(ORBClientCoreMessageHandler.java:328)
at com.ibm.ws.objectgrid.client.ORBClientCoreMessageHandler.sendReadWriteRequest(ORBClientCoreMessageHandler.java:1044)
at com.ibm.ws.objectgrid.client.RemoteTransactionCallbackImpl.processReadWriteRequestAndResponse(RemoteTransactionCallbackImpl.java:970)
at com.ibm.ws.objectgrid.client.RemoteTransactionCallbackImpl.commit(RemoteTransactionCallbackImpl.java:283)
at com.ibm.ws.objectgrid.SessionImpl.commit(SessionImpl.java:1848)
at com.ibm.ws.xs.sessionmanager.SessionReplicationManager.run(SessionReplicationManager.java:719)
at java.util.Timer$TimerImpl.run(Timer.java:296)
Caused by: java.io.NotSerializableException: com.ibm.wps.engine.PortalRequestWrapper
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1538)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1503)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1420)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)

==

Can anyone let me know what is the root cause of this issue.

FYI..I have followed the following URL for enabling Xtreme Scale Cache.

http://www-01.ibm.com/support/docview.wss?uid=swg21619042
Updated on 2013-04-04T11:51:18Z at 2013-04-04T11:51:18Z by SystemAdmin
  • joshuad
    joshuad
    38 Posts

    Re: Serializable Issue ?

    ‏2013-04-02T15:00:03Z  
    For which applications have you enabled WXS session replication? Portal has several applications that do not support session replication (due to serialization issues), so you should only enable specifically the wps application as well as any custom portlets you have created that store serializable attributes. Can you verify WXS session replication is only set at the application level, and only for wps and your custom apps?
  • SystemAdmin
    SystemAdmin
    1485 Posts

    Re: Serializable Issue ?

    ‏2013-04-03T08:06:51Z  
    • joshuad
    • ‏2013-04-02T15:00:03Z
    For which applications have you enabled WXS session replication? Portal has several applications that do not support session replication (due to serialization issues), so you should only enable specifically the wps application as well as any custom portlets you have created that store serializable attributes. Can you verify WXS session replication is only set at the application level, and only for wps and your custom apps?
    Thanks for your reply.

    I have enabled the the Session Replication at the server level (i.e. in the WebSphere application servers level).

    I would like to know how does enabling session replication at server level different than enabling from the wps/custom application level?

    Can you provide some pointers for the same and also any other parameters as well

    Thanks...
  • joshuad
    joshuad
    38 Posts

    Re: Serializable Issue ?

    ‏2013-04-03T13:28:12Z  
    Thanks for your reply.

    I have enabled the the Session Replication at the server level (i.e. in the WebSphere application servers level).

    I would like to know how does enabling session replication at server level different than enabling from the wps/custom application level?

    Can you provide some pointers for the same and also any other parameters as well

    Thanks...
    When you enable WXS session replication at the server level, all applications installed on that server will be configured to replicate session data to the remote grid. When using Portal, due to serialization issues with their applications, you must enable WXS at the application level. In the WAS admin console, Enterprise Applications -> <app name> -> Session management -> eXtreme Scale session management settings. Enable session management and input the proper catalog service domain and session grid. Do this for the wps.ear as well as your custom applications. Then disable the server level session replication. Restart the portal server. That should take care of the serialization issues.
  • SystemAdmin
    SystemAdmin
    1485 Posts

    Re: Serializable Issue ?

    ‏2013-04-04T11:51:18Z  
    • joshuad
    • ‏2013-04-03T13:28:12Z
    When you enable WXS session replication at the server level, all applications installed on that server will be configured to replicate session data to the remote grid. When using Portal, due to serialization issues with their applications, you must enable WXS at the application level. In the WAS admin console, Enterprise Applications -> <app name> -> Session management -> eXtreme Scale session management settings. Enable session management and input the proper catalog service domain and session grid. Do this for the wps.ear as well as your custom applications. Then disable the server level session replication. Restart the portal server. That should take care of the serialization issues.
    Thanks for your reply...
  • Dev_Dhoot
    Dev_Dhoot
    42 Posts

    Re: Serializable Issue ?

    ‏2013-07-11T09:22:41Z  
    • joshuad
    • ‏2013-04-03T13:28:12Z
    When you enable WXS session replication at the server level, all applications installed on that server will be configured to replicate session data to the remote grid. When using Portal, due to serialization issues with their applications, you must enable WXS at the application level. In the WAS admin console, Enterprise Applications -> <app name> -> Session management -> eXtreme Scale session management settings. Enable session management and input the proper catalog service domain and session grid. Do this for the wps.ear as well as your custom applications. Then disable the server level session replication. Restart the portal server. That should take care of the serialization issues.

    Hi Joshuad,

    I am facing the similar issue. I read this post and have an question here.

     

    Can you let me know how this solution would fit in case of multiple Virtual Portal existing in a Portal environment. I have couple of Virtual Portals in my Portal architecture and would like to implement WXS caching on a specific virtual portal only.

    These all Virtual Portals are sharing some common Base Portal resources.

    Does enabling the session on wps.ear would invoke it on all the virtual portals as well.

    Can you let me know the best practice for the same and also highlight the correct mechanism for the architecture as specified above.

     

    Thanks.

    --Devendra

     

  • Dev_Dhoot
    Dev_Dhoot
    42 Posts

    Re: Serializable Issue ?

    ‏2013-07-12T09:26:40Z  
    • joshuad
    • ‏2013-04-03T13:28:12Z
    When you enable WXS session replication at the server level, all applications installed on that server will be configured to replicate session data to the remote grid. When using Portal, due to serialization issues with their applications, you must enable WXS at the application level. In the WAS admin console, Enterprise Applications -> <app name> -> Session management -> eXtreme Scale session management settings. Enable session management and input the proper catalog service domain and session grid. Do this for the wps.ear as well as your custom applications. Then disable the server level session replication. Restart the portal server. That should take care of the serialization issues.

    Hi,

     

    I have gone through your post and enabled the WXS caching at the portlet level. This means I have enabled caching at my Custom application along with wps.ear and then  restarted the Portal server.

    The issue I am facing now is hung threads.

     

    Below is the snippet of the SystemOut.log.

    ==

    Caused by: com.ibm.ws.xsspi.xio.exception.MessageTimeOutException [originating=x.x.x.x:0;causedby=*.*.*.*:4009;exid=235]: com.ibm.ws.xs.xio.protobuf.ContainerMessages$GetRequestMessage/reqID=24879 await timeout after 30000 ms contacting *.*.*.*:4009 queue size on insert=138, waiter {id: 8312, index: 14}
     
     
    [7/11/13 8:12:52:292 CDT] 00000023 ThreadMonitor W   WSVR0605W: Thread "WebContainer : 11" (0000005e) has been active for 610161 milliseconds and may be hung.  There is/are 7 thread(s) in total in the server that may be hung.
            at java.lang.Object.wait(Native Method)
            at java.lang.Object.wait(Object.java:167)
            at com.ibm.ws.xs.sessionmanager.SessionsLRUOffLoader.checkAndWaitLockKey(SessionsLRUOffLoader.java:284)
            at com.ibm.ws.xs.sessionmanager.SessionsLRUOffLoader.accessObject(SessionsLRUOffLoader.java:557)
            at com.ibm.ws.xs.sessionmanager.HttpSessionRequestWrapper.getSession(HttpSessionRequestWrapper.java:690)
            at javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:238)
            at javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:238)
            at javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:238)
            at javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:238)
            at com.ibm.wps.engine.PortalRequestWrapper.getSession(PortalRequestWrapper.java:805)
            at com.ibm.wps.services.authentication.impl.AuthenticationServiceImpl.handleAppServerSessionSecurity(AuthenticationServiceImpl.java:1194)
            at com.ibm.wps.engine.Servlet.doGet(Servlet.java:500)
            at com.ibm.wps.engine.Servlet.doPost(Servlet.java:852)
            at javax.servlet.http.HttpServlet.service(HttpServlet.java:595)
            at com.ibm.wps.engine.Servlet.doFilter(Servlet.java:411)
            at com.ibm.wps.resolver.servlet.ContentHandlerCleanup.doFilter(ContentHandlerCleanup.java:780)
            at com.ibm.wps.resolver.servlet.AbstractFilter.doFilter(AbstractFilter.java:96)
            at com.ibm.wps.engine.Servlet.service(Servlet.java:1374)

    ==

    Environment WebSphere Extreme Scale 8.6.0.2 + Interim Fix

                                    WebSphere Portal Server 8.0.0.1 + CF 5

     

    Can anyone let me know what could be done to solve the issue?

    Updated on 2013-07-12T09:28:02Z at 2013-07-12T09:28:02Z by Dev_Dhoot
  • joshuad
    joshuad
    38 Posts

    Re: Serializable Issue ?

    ‏2013-07-12T12:28:53Z  
    • Dev_Dhoot
    • ‏2013-07-12T09:26:40Z

    Hi,

     

    I have gone through your post and enabled the WXS caching at the portlet level. This means I have enabled caching at my Custom application along with wps.ear and then  restarted the Portal server.

    The issue I am facing now is hung threads.

     

    Below is the snippet of the SystemOut.log.

    ==

    Caused by: com.ibm.ws.xsspi.xio.exception.MessageTimeOutException [originating=x.x.x.x:0;causedby=*.*.*.*:4009;exid=235]: com.ibm.ws.xs.xio.protobuf.ContainerMessages$GetRequestMessage/reqID=24879 await timeout after 30000 ms contacting *.*.*.*:4009 queue size on insert=138, waiter {id: 8312, index: 14}
     
     
    [7/11/13 8:12:52:292 CDT] 00000023 ThreadMonitor W   WSVR0605W: Thread "WebContainer : 11" (0000005e) has been active for 610161 milliseconds and may be hung.  There is/are 7 thread(s) in total in the server that may be hung.
            at java.lang.Object.wait(Native Method)
            at java.lang.Object.wait(Object.java:167)
            at com.ibm.ws.xs.sessionmanager.SessionsLRUOffLoader.checkAndWaitLockKey(SessionsLRUOffLoader.java:284)
            at com.ibm.ws.xs.sessionmanager.SessionsLRUOffLoader.accessObject(SessionsLRUOffLoader.java:557)
            at com.ibm.ws.xs.sessionmanager.HttpSessionRequestWrapper.getSession(HttpSessionRequestWrapper.java:690)
            at javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:238)
            at javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:238)
            at javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:238)
            at javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:238)
            at com.ibm.wps.engine.PortalRequestWrapper.getSession(PortalRequestWrapper.java:805)
            at com.ibm.wps.services.authentication.impl.AuthenticationServiceImpl.handleAppServerSessionSecurity(AuthenticationServiceImpl.java:1194)
            at com.ibm.wps.engine.Servlet.doGet(Servlet.java:500)
            at com.ibm.wps.engine.Servlet.doPost(Servlet.java:852)
            at javax.servlet.http.HttpServlet.service(HttpServlet.java:595)
            at com.ibm.wps.engine.Servlet.doFilter(Servlet.java:411)
            at com.ibm.wps.resolver.servlet.ContentHandlerCleanup.doFilter(ContentHandlerCleanup.java:780)
            at com.ibm.wps.resolver.servlet.AbstractFilter.doFilter(AbstractFilter.java:96)
            at com.ibm.wps.engine.Servlet.service(Servlet.java:1374)

    ==

    Environment WebSphere Extreme Scale 8.6.0.2 + Interim Fix

                                    WebSphere Portal Server 8.0.0.1 + CF 5

     

    Can anyone let me know what could be done to solve the issue?

    Can you reproduce the issue with "session=all" trace enabled on the portal server?  If so, please attach the logs directory of the Portal server zipped up and I will take a look:

    Stop the portal server

    enable "*=info: session=all" trace on the portal server and restart

    run as few requests as possible to reproduce the failure

    Attach the resulting logs directory of the portal server zipped up

     

    Thanks.

  • Dev_Dhoot
    Dev_Dhoot
    42 Posts

    Re: Serializable Issue ?

    ‏2013-07-15T12:06:16Z  
    • joshuad
    • ‏2013-07-12T12:28:53Z

    Can you reproduce the issue with "session=all" trace enabled on the portal server?  If so, please attach the logs directory of the Portal server zipped up and I will take a look:

    Stop the portal server

    enable "*=info: session=all" trace on the portal server and restart

    run as few requests as possible to reproduce the failure

    Attach the resulting logs directory of the portal server zipped up

     

    Thanks.

    Joshuad,

     

    Thanks for the reply.

    I have followed the above steps and collected the logs for the same timeframe.

    Please find attached logs in teh zip format.

    Please let me know if you need anything else from my side.

     

    --Devendra

     

    Attachments

  • joshuad
    joshuad
    38 Posts

    Re: Serializable Issue ?

    ‏2013-07-15T16:27:02Z  
    • Dev_Dhoot
    • ‏2013-07-15T12:06:16Z

    Joshuad,

     

    Thanks for the reply.

    I have followed the above steps and collected the logs for the same timeframe.

    Please find attached logs in teh zip format.

    Please let me know if you need anything else from my side.

     

    --Devendra

     

    Please try this fix for your problem.  You'll need to stop your portal servers, put this jar in your WAS_HOME/lib directory (backup the jar that's already there), restart the portal servers and retry your scenario.  Please let me know if this works for you.

    Attachments

  • Dev_Dhoot
    Dev_Dhoot
    42 Posts

    Re: Serializable Issue ?

    ‏2013-07-16T14:28:06Z  
    • joshuad
    • ‏2013-07-15T16:27:02Z

    Please try this fix for your problem.  You'll need to stop your portal servers, put this jar in your WAS_HOME/lib directory (backup the jar that's already there), restart the portal servers and retry your scenario.  Please let me know if this works for you.

    Hi Joshuad,

    Appreciate your quick response on this!!

     

    I have replaced the "sessionobjectgrid.jar" and reproduced the scenario. The Hung thread exceptions are  now gone!!

    The primary exception is the below one:

    ==

    com.ibm.ws.webcontainer.filter.FilterInstanceWrapper doFilter SRVE8109W: Uncaught exception thrown by filter ZipHandler: java.io.FileNotFoundException: SRVE0190E: File not found: /v1.7/dojo/nls/dojo_en.js
    at com.ibm.ws.webcontainer.extension.DefaultExtensionProcessor._processEDR(DefaultExtensionProcessor.java:893)
    at com.ibm.ws.webcontainer.extension.DefaultExtensionProcessor.processEDR(DefaultExtensionProcessor.java:874)
    at com.ibm.ws.webcontainer.extension.DefaultExtensionProcessor.handleRequest(DefaultExtensionProcessor.java:435)
    at com.ibm.ws.webcontainer.filter.WebAppFilterChain.invokeTarget(WebAppFilterChain.java:136)
    at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:97)
    at com.ibm.wps.resolver.servlet.ContentHandlerGzip.internalDoFilter(ContentHandlerGzip.java:704)
    at com.ibm.wps.resolver.servlet.ContentHandlerGzip.doFilter(ContentHandlerGzip.java:451)
    at com.ibm.wps.resolver.servlet.AbstractFilter.doFilter(AbstractFilter.java:96)
    at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:195)
    ==

    I have attached relevant logs generated for the particular time frame. 

    The dojo exceptions are frequently seen in the logs and are causing some application functionality to break.

    Also can you let me know the root cause behind the hung thread exception, so that it can be avoided in the future.

    Thanks!

    --Devendra

     

    Attachments

    Updated on 2013-07-16T14:29:31Z at 2013-07-16T14:29:31Z by Dev_Dhoot
  • joshuad
    joshuad
    38 Posts

    Re: Serializable Issue ?

    ‏2013-07-16T16:53:01Z  
    • Dev_Dhoot
    • ‏2013-07-16T14:28:06Z

    Hi Joshuad,

    Appreciate your quick response on this!!

     

    I have replaced the "sessionobjectgrid.jar" and reproduced the scenario. The Hung thread exceptions are  now gone!!

    The primary exception is the below one:

    ==

    com.ibm.ws.webcontainer.filter.FilterInstanceWrapper doFilter SRVE8109W: Uncaught exception thrown by filter ZipHandler: java.io.FileNotFoundException: SRVE0190E: File not found: /v1.7/dojo/nls/dojo_en.js
    at com.ibm.ws.webcontainer.extension.DefaultExtensionProcessor._processEDR(DefaultExtensionProcessor.java:893)
    at com.ibm.ws.webcontainer.extension.DefaultExtensionProcessor.processEDR(DefaultExtensionProcessor.java:874)
    at com.ibm.ws.webcontainer.extension.DefaultExtensionProcessor.handleRequest(DefaultExtensionProcessor.java:435)
    at com.ibm.ws.webcontainer.filter.WebAppFilterChain.invokeTarget(WebAppFilterChain.java:136)
    at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:97)
    at com.ibm.wps.resolver.servlet.ContentHandlerGzip.internalDoFilter(ContentHandlerGzip.java:704)
    at com.ibm.wps.resolver.servlet.ContentHandlerGzip.doFilter(ContentHandlerGzip.java:451)
    at com.ibm.wps.resolver.servlet.AbstractFilter.doFilter(AbstractFilter.java:96)
    at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:195)
    ==

    I have attached relevant logs generated for the particular time frame. 

    The dojo exceptions are frequently seen in the logs and are causing some application functionality to break.

    Also can you let me know the root cause behind the hung thread exception, so that it can be avoided in the future.

    Thanks!

    --Devendra

     

    The dojo issue looks to be a known problem unrelated to eXtreme Scale.  Please take a look at the following link and hopefully it will help:  https://www.ibm.com/developerworks/community/forums/html/topic?id=77777777-0000-0000-0000-000014880367

     

    The hung thread issue was simply a bug in the WXS client code which has been fixed with the jar. There was a certain path that Portal is more likely to go down with how it uses sessions, and that path was not properly releasing certain locks, which caused the hung threads.  The official fix will be available in the near future in the next fixpack for WXS 8.6

  • Dev_Dhoot
    Dev_Dhoot
    42 Posts

    Re: Serializable Issue ?

    ‏2013-07-17T12:07:41Z  
    • joshuad
    • ‏2013-07-16T16:53:01Z

    The dojo issue looks to be a known problem unrelated to eXtreme Scale.  Please take a look at the following link and hopefully it will help:  https://www.ibm.com/developerworks/community/forums/html/topic?id=77777777-0000-0000-0000-000014880367

     

    The hung thread issue was simply a bug in the WXS client code which has been fixed with the jar. There was a certain path that Portal is more likely to go down with how it uses sessions, and that path was not properly releasing certain locks, which caused the hung threads.  The official fix will be available in the near future in the next fixpack for WXS 8.6

    I applied the changes as mentioned in your earlier  post and tested the functionality, but it didn't work.

    I was not seeing the dojo exception before the WXS was enabled in the Portal server.  I guess, this might be the root cause behind  the functionality to break. If I remove the WXS session configuration, the functionality works. 

    Ex: When I traverse on a IBM -default page (Portal 8 theme) and select the Edit Page option, it spins forever and doesn't open the fly-out, the logs shows dojo exception.

    Can you guide me what went wrong when WXS session configuration is enabled?

    --Devendra

  • joshuad
    joshuad
    38 Posts

    Re: Serializable Issue ?

    ‏2013-07-18T14:59:05Z  
    • Dev_Dhoot
    • ‏2013-07-17T12:07:41Z

    I applied the changes as mentioned in your earlier  post and tested the functionality, but it didn't work.

    I was not seeing the dojo exception before the WXS was enabled in the Portal server.  I guess, this might be the root cause behind  the functionality to break. If I remove the WXS session configuration, the functionality works. 

    Ex: When I traverse on a IBM -default page (Portal 8 theme) and select the Edit Page option, it spins forever and doesn't open the fly-out, the logs shows dojo exception.

    Can you guide me what went wrong when WXS session configuration is enabled?

    --Devendra

    I don't see any problems from the provided trace. I can take a look at a WebSphere trace as well if you'd like to rerun:

    com.ibm.ws.session.*=all:com.ibm.ws.webcontainer.srt.*=all:session=all

    Set that trace on your portal servers, and attach the traces.

  • Dev_Dhoot
    Dev_Dhoot
    42 Posts

    Re: Serializable Issue ?

    ‏2013-07-22T12:42:50Z  
    • joshuad
    • ‏2013-07-18T14:59:05Z

    I don't see any problems from the provided trace. I can take a look at a WebSphere trace as well if you'd like to rerun:

    com.ibm.ws.session.*=all:com.ibm.ws.webcontainer.srt.*=all:session=all

    Set that trace on your portal servers, and attach the traces.

    I have enabled the traces as mentioned in your previous post and then reproduced  the scenario.

    Here is the attached logs folder.  I hope this would be useful in identifying the root cause.

     

    --Devendra

    Attachments

  • joshuad
    joshuad
    38 Posts

    Re: Serializable Issue ?

    ‏2013-07-22T16:45:26Z  
    • Dev_Dhoot
    • ‏2013-07-22T12:42:50Z

    I have enabled the traces as mentioned in your previous post and then reproduced  the scenario.

    Here is the attached logs folder.  I hope this would be useful in identifying the root cause.

     

    --Devendra

    The last attached logs still show the following trace spec:  [7/22/13 5:45:51:253 CDT] 00000000 ManagerAdmin  I   TRAS0017I: The startup trace state is *=info:session=all.

     

    Please retry with the previously mentioned trace spec:  com.ibm.ws.session.*=all:com.ibm.ws.webcontainer.srt.*=all:session=all

  • Dev_Dhoot
    Dev_Dhoot
    42 Posts

    Re: Serializable Issue ?

    ‏2013-07-23T13:07:17Z  
    • joshuad
    • ‏2013-07-22T16:45:26Z

    The last attached logs still show the following trace spec:  [7/22/13 5:45:51:253 CDT] 00000000 ManagerAdmin  I   TRAS0017I: The startup trace state is *=info:session=all.

     

    Please retry with the previously mentioned trace spec:  com.ibm.ws.session.*=all:com.ibm.ws.webcontainer.srt.*=all:session=all

    These are the updated logs with the trace settings enabled. I hope these would be helpful in identifying the root cause.

     

    --Devendra

     

    Attachments

  • joshuad
    joshuad
    38 Posts

    Re: Serializable Issue ?

    ‏2013-07-23T16:42:40Z  
    • Dev_Dhoot
    • ‏2013-07-23T13:07:17Z

    These are the updated logs with the trace settings enabled. I hope these would be helpful in identifying the root cause.

     

    --Devendra

     

    The trace isn't showing any problems with WXS HttpSession.  You could try to enable the Dojo_Resources application to use the same grid that you're using with your other portlets.  Otherwise would have to ask the Portal experts what could be causing the FileNotFoundExceptions.

     

    Have you also followed this step in your configuration?

     

    1. Some versions of WebSphere Portal server can have runtime errors when cookies are added to an HTTP response. Since adds cookies for failover and other purposes, these cookies need to be added to WebSphere Portal server cookie ignore list. For more information, see the cookie.ignore.regex parameter section of Caching pages shared by multiple users on the IBM WebSphere Portal wiki. The two cookies that need to be added to the list are IBMID.* and IBMSessionHandle.*. The updated list may look like this for example"digest\\.ignore.*|LtpaToken|LtpaToken2|JSESSIONID|IBMID.*|IBMSessionHandle.*". For more information, see Caching pages shared by multiple users on the IBM WebSphere Portal wiki.
  • chuck1024
    chuck1024
    1 Post

    Re: Serializable Issue ?

    ‏2013-10-28T18:56:51Z  
    • joshuad
    • ‏2013-07-15T16:27:02Z

    Please try this fix for your problem.  You'll need to stop your portal servers, put this jar in your WAS_HOME/lib directory (backup the jar that's already there), restart the portal servers and retry your scenario.  Please let me know if this works for you.

    We may be having the same issue, where can we see an article about this fix and download a patch?

     

    Running 8.5.0.3 with error messages like this:
     

    [10/27/13 23:13:01:672 EDT] 00000003 ThreadMonitor W   WSVR0605W: Thread "WebContainer : 24" (000000ca) has been active for 681471 milliseconds and may be hu
    ng.  There is/are 22 thread(s) in total in the server that may be hung.
            at java.lang.Object.wait(Native Method)
            at java.lang.Object.wait(Object.java:167)
            at com.ibm.ws.xs.sessionmanager.SessionsLRUOffLoader.checkAndWaitLockKey(SessionsLRUOffLoader.java:285)
            at com.ibm.ws.xs.sessionmanager.SessionsLRUOffLoader.accessObject(SessionsLRUOffLoader.java:560)
            at com.ibm.ws.xs.sessionmanager.HttpSessionRequestWrapper.getSession(HttpSessionRequestWrapper.java:648)
            at com.ibm.ws.xs.sessionmanager.HttpSessionRequestWrapper.getSession(HttpSessionRequestWrapper.java:311)