Topic
  • 8 replies
  • Latest Post - ‏2013-05-01T12:24:21Z by jhanders
ibm_extremescale
ibm_extremescale
1 Post

Pinned topic Null value getting stored in object map

‏2012-07-02T18:51:39Z |
Hi,
We are encountering a scenario were a null value is stored in certain object maps for few keys (userID).
When we invalidate the key using remove() method in extreme scale we are receiving java.lang.NullPointerException as the key already has null value stored in it. Would like to know how those keys containing null could be removed without code changes preferably through console.
Already recycled the object grid, extreme scale components but maps still hold the null.

Exception Snippet:
7/2/12 11:56:11:034 EDT 0000003e WXSClientCach I Exception in invalidateUserProfileAndEntitlements - WXSClientCache : com.ibm.websphere.objectgrid.TransactionException: rolling back transaction, see caused by exception
com.ibm.websphere.objectgrid.ObjectGridRuntimeException: com.ibm.websphere.objectgrid.TransactionException: rolling back transaction, see caused by exception
at com.bnym.wxs.utils.BNYMWXSMap.remove(BNYMWXSMap.java:121)
at com.bnym.wxs.client.WXSClientCache.invalidateUserProfileAndEntitlements(WXSClientCache.java:609)
at com.bnym.wxs.web.WXSWebClientController.invalidate(WXSWebClientController.java:201)
at com.bnym.wxs.web.WXSWebClientController.doGet(WXSWebClientController.java:102)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:743)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1075)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:550)
at com.ibm.ws.wswebcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:478)
at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:90)
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:744)
at com.ibm.ws.wswebcontainer.WebContainer.handleRequest(WebContainer.java:1455)
at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:115)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:458)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewInformation(HttpInboundLink.java:387)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.ready(HttpInboundLink.java:267)
at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.sendToDiscriminators(NewConnectionInitialReadCallback.java:214)
at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.complete(NewConnectionInitialReadCallback.java:113)
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:165)
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:136)
at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:195)
at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:743)
at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:873)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1473)
Caused by: com.ibm.websphere.objectgrid.TransactionException: rolling back transaction, see caused by exception
at com.ibm.ws.objectgrid.SessionImpl.rollbackPMapChanges(SessionImpl.java:1558)
at com.ibm.ws.objectgrid.SessionImpl.commit(SessionImpl.java:1333)
at com.ibm.ws.objectgrid.SessionImpl.mapPostInvoke(SessionImpl.java:2510)
at com.ibm.ws.objectgrid.ObjectMapImpl.remove(ObjectMapImpl.java:728)
at com.ibm.ws.objectgrid.ObjectMapImpl.remove(ObjectMapImpl.java:680)
at com.bnym.wxs.utils.BNYMWXSMap.remove(BNYMWXSMap.java:112)
... 25 more
Caused by: com.ibm.websphere.objectgrid.TransactionException: Transaction was previously marked as rollback only
at com.ibm.ws.objectgrid.SessionImpl.commit(SessionImpl.java:1059)
... 29 more
Caused by: com.ibm.websphere.objectgrid.ObjectGridRuntimeException: java.lang.NullPointerException
at com.ibm.ws.objectgrid.map.BaseMap.bytesToValue(BaseMap.java:8854)
at com.ibm.ws.objectgrid.map.BaseMap.getValueCopyForTransaction(BaseMap.java:3460)
at com.ibm.ws.objectgrid.DiffMapValue.getCurrentValue(DiffMapValue.java:394)
at com.ibm.ws.objectgrid.DiffMap.removeFromDiffMap(DiffMap.java:1537)
at com.ibm.ws.objectgrid.DiffMap.remove(DiffMap.java:1487)
at com.ibm.ws.objectgrid.ObjectMapImpl.remove(ObjectMapImpl.java:707)
... 27 more
Caused by: java.lang.NullPointerException
at com.ibm.ws.objectgrid.map.BaseMap$BaseMapObjectTransformer2.inflateObject(BaseMap.java:4807)
at com.ibm.ws.objectgrid.map.BaseMap$BaseMapObjectTransformer.inflateValue(BaseMap.java:4689)
at com.ibm.ws.objectgrid.map.BaseMap.bytesToValue(BaseMap.java:8849)
... 32 more

Thanks in advance !!!
Updated on 2012-07-03T17:27:47Z at 2012-07-03T17:27:47Z by jhanders
  • jhanders
    jhanders
    265 Posts

    Re: Null value getting stored in object map

    ‏2012-07-03T17:27:47Z  
    Based off of the stack you are using WebSphere eXtreme Scale 7.0. In this level of the product, if the internal bytehttp://] that we are storing when using copy to bytes is null you will get this exception. In 7.1 we have updated it to return null if the byte[ is null. The byte[] is not supposed to be null though. For it to be null it means that something likely happened when we were trying to convert the POJO to a byte array. You should expect that an exception happened early when the entry was inserted that would cause this NullPointerException to happen on the remove operation. You should look at your FFDC and SystemOut to see if something is logged there for an earlier problem. You could also try your scenario on 7.1.1.1 or 8.5.0.1 to see if it helps with diagnosing the problem since we have updated several paths to have it fail instead of blindly setting it to null like was done in 7.0.

    I hope that helps.

    Jared Anderson
  • Dev_Dhoot
    Dev_Dhoot
    42 Posts

    Re: Null value getting stored in object map

    ‏2013-04-25T07:52:45Z  
    • jhanders
    • ‏2012-07-03T17:27:47Z
    Based off of the stack you are using WebSphere eXtreme Scale 7.0. In this level of the product, if the internal bytehttp://] that we are storing when using copy to bytes is null you will get this exception. In 7.1 we have updated it to return null if the byte[ is null. The byte[] is not supposed to be null though. For it to be null it means that something likely happened when we were trying to convert the POJO to a byte array. You should expect that an exception happened early when the entry was inserted that would cause this NullPointerException to happen on the remove operation. You should look at your FFDC and SystemOut to see if something is logged there for an earlier problem. You could also try your scenario on 7.1.1.1 or 8.5.0.1 to see if it helps with diagnosing the problem since we have updated several paths to have it fail instead of blindly setting it to null like was done in 7.0.

    I hope that helps.

    Jared Anderson

    James,

     

    I am facing a similar issue on WXS 8.5 

    Below are the error snippet.

     

    ==

    Caused by: com.ibm.websphere.objectgrid.ObjectGridRuntimeException: java.io.StreamCorruptedException: invalid type code: 3F 
    at com.ibm.ws.objectgrid.map.BaseMap.bytesToObject(BaseMap.java:12127) ~[wsogclient.jar:WXS8.5.0.XS [a1216.20150800]] 
    at com.ibm.ws.objectgrid.map.BaseMap.getValueCopyForTransaction(BaseMap.java:4763) ~[wsogclient.jar:WXS8.5.0.XS [a1216.20150800]] 
    at com.ibm.ws.objectgrid.map.BaseMap.getValueCopyForTransaction(BaseMap.java:4720) ~[wsogclient.jar:WXS8.5.0.XS [a1216.20150800]] 
    at com.ibm.ws.objectgrid.DiffMapValue.getCurrentValue(DiffMapValue.java:800) ~[wsogclient.jar:WXS8.5.0.XS [a1216.20150800]] 
    at com.ibm.ws.objectgrid.DiffMap.get(DiffMap.java:952) ~[wsogclient.jar:WXS8.5.0.XS [a1216.20150800]] 
    at com.ibm.ws.objectgrid.ObjectMapImpl.get(ObjectMapImpl.java:408) ~[wsogclient.jar:WXS8.5.0.XS [a1216.20150800]] 
    ... 146 common frames omitted 
    Caused by: java.io.StreamCorruptedException: invalid type code: 3F 
    at java.io.ObjectInputStream$BlockDataInputStream.readBlockHeader(ObjectInputStream.java:2481) ~[na:1.6.0] 
    at java.io.ObjectInputStream$BlockDataInputStream.refill(ObjectInputStream.java:2516) ~[na:1.6.0] 
    at java.io.ObjectInputStream$BlockDataInputStream.read(ObjectInputStream.java:2588) ~[na:1.6.0] 
    at java.io.ObjectInputStream$BlockDataInputStream.readBoolean(ObjectInputStream.java:2729) ~[na:1.6.0] 
    at java.io.ObjectInputStream.readBoolean(ObjectInputStream.java:898) ~[na:1.6.0] 
    at com.ibm.ws.xs.stats.datamodel.XSStatsStore.readExternal(XSStatsStore.java:73) ~[wsogclient.jar:WXS8.5.0.XS [a1216.20150800]] 
    at com.ibm.ws.xs.stats.datamodel.MapStatsStore.readExternal(MapStatsStore.java:104) ~[wsogclient.jar:WXS8.5.0.XS [a1216.20150800]] 
    at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:1809) ~[na:1.6.0] 
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1768) ~[na:1.6.0] 
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1346) ~[na:1.6.0] 
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964) ~[na:1.6.0] 
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888) ~[na:1.6.0] 
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1770) ~[na:1.6.0] 
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1346) ~[na:1.6.0] 
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:365) ~[na:1.6.0] 
    at com.ibm.ws.objectgrid.plugins.GenericSerializationInfo.inflate(GenericSerializationInfo.java:84) ~[wsogclient.jar:WXS8.5.0.XS [a1216.20150800]] 
    at com.ibm.ws.objectgrid.map.BaseMap$BaseMapObjectTransformer2.inflateObject(BaseMap.java:6599) ~[wsogclient.jar:WXS8.5.0.XS [a1216.20150800]] 
    at com.ibm.ws.objectgrid.map.BaseMap$BaseMapObjectTransformer.inflateValue(BaseMap.java:6440) ~[wsogclient.jar:WXS8.5.0.XS [a1216.20150800]] 
    at com.ibm.ws.objectgrid.map.BaseMap.bytesToObject(BaseMap.java:12046) ~[wsogclient.jar:WXS8.5.0.XS [a1216.20150800]] 
    ... 151 common frames omitted 

    ==

     

    Can you please let me know how to fix this?

     

    Thanks.

  • jhanders
    jhanders
    265 Posts

    Re: Null value getting stored in object map

    ‏2013-04-25T10:22:10Z  
    • Dev_Dhoot
    • ‏2013-04-25T07:52:45Z

    James,

     

    I am facing a similar issue on WXS 8.5 

    Below are the error snippet.

     

    ==

    Caused by: com.ibm.websphere.objectgrid.ObjectGridRuntimeException: java.io.StreamCorruptedException: invalid type code: 3F 
    at com.ibm.ws.objectgrid.map.BaseMap.bytesToObject(BaseMap.java:12127) ~[wsogclient.jar:WXS8.5.0.XS [a1216.20150800]] 
    at com.ibm.ws.objectgrid.map.BaseMap.getValueCopyForTransaction(BaseMap.java:4763) ~[wsogclient.jar:WXS8.5.0.XS [a1216.20150800]] 
    at com.ibm.ws.objectgrid.map.BaseMap.getValueCopyForTransaction(BaseMap.java:4720) ~[wsogclient.jar:WXS8.5.0.XS [a1216.20150800]] 
    at com.ibm.ws.objectgrid.DiffMapValue.getCurrentValue(DiffMapValue.java:800) ~[wsogclient.jar:WXS8.5.0.XS [a1216.20150800]] 
    at com.ibm.ws.objectgrid.DiffMap.get(DiffMap.java:952) ~[wsogclient.jar:WXS8.5.0.XS [a1216.20150800]] 
    at com.ibm.ws.objectgrid.ObjectMapImpl.get(ObjectMapImpl.java:408) ~[wsogclient.jar:WXS8.5.0.XS [a1216.20150800]] 
    ... 146 common frames omitted 
    Caused by: java.io.StreamCorruptedException: invalid type code: 3F 
    at java.io.ObjectInputStream$BlockDataInputStream.readBlockHeader(ObjectInputStream.java:2481) ~[na:1.6.0] 
    at java.io.ObjectInputStream$BlockDataInputStream.refill(ObjectInputStream.java:2516) ~[na:1.6.0] 
    at java.io.ObjectInputStream$BlockDataInputStream.read(ObjectInputStream.java:2588) ~[na:1.6.0] 
    at java.io.ObjectInputStream$BlockDataInputStream.readBoolean(ObjectInputStream.java:2729) ~[na:1.6.0] 
    at java.io.ObjectInputStream.readBoolean(ObjectInputStream.java:898) ~[na:1.6.0] 
    at com.ibm.ws.xs.stats.datamodel.XSStatsStore.readExternal(XSStatsStore.java:73) ~[wsogclient.jar:WXS8.5.0.XS [a1216.20150800]] 
    at com.ibm.ws.xs.stats.datamodel.MapStatsStore.readExternal(MapStatsStore.java:104) ~[wsogclient.jar:WXS8.5.0.XS [a1216.20150800]] 
    at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:1809) ~[na:1.6.0] 
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1768) ~[na:1.6.0] 
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1346) ~[na:1.6.0] 
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964) ~[na:1.6.0] 
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888) ~[na:1.6.0] 
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1770) ~[na:1.6.0] 
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1346) ~[na:1.6.0] 
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:365) ~[na:1.6.0] 
    at com.ibm.ws.objectgrid.plugins.GenericSerializationInfo.inflate(GenericSerializationInfo.java:84) ~[wsogclient.jar:WXS8.5.0.XS [a1216.20150800]] 
    at com.ibm.ws.objectgrid.map.BaseMap$BaseMapObjectTransformer2.inflateObject(BaseMap.java:6599) ~[wsogclient.jar:WXS8.5.0.XS [a1216.20150800]] 
    at com.ibm.ws.objectgrid.map.BaseMap$BaseMapObjectTransformer.inflateValue(BaseMap.java:6440) ~[wsogclient.jar:WXS8.5.0.XS [a1216.20150800]] 
    at com.ibm.ws.objectgrid.map.BaseMap.bytesToObject(BaseMap.java:12046) ~[wsogclient.jar:WXS8.5.0.XS [a1216.20150800]] 
    ... 151 common frames omitted 

    ==

     

    Can you please let me know how to fix this?

     

    Thanks.

    The issue you are hitting is the problem that we fixed with APAR PM76020.  On the 8.5 level this APAR is included in 8.5.0.3.  Your data above indicates that you are running the GA version of 8.5.  There has been a lot of service in the last year for 8.5 so I would recommend that you move up to 8.5.0.3 to resolve your issue.  There are changes for both the client and server code for this fix so make sure you have both updated to the 8.5.0.3 level.

    Jared Anderson

  • Dev_Dhoot
    Dev_Dhoot
    42 Posts

    Re: Null value getting stored in object map

    ‏2013-04-25T10:54:28Z  
    • jhanders
    • ‏2013-04-25T10:22:10Z

    The issue you are hitting is the problem that we fixed with APAR PM76020.  On the 8.5 level this APAR is included in 8.5.0.3.  Your data above indicates that you are running the GA version of 8.5.  There has been a lot of service in the last year for 8.5 so I would recommend that you move up to 8.5.0.3 to resolve your issue.  There are changes for both the client and server code for this fix so make sure you have both updated to the 8.5.0.3 level.

    Jared Anderson

    Jared,

    Thanks for the reply.

    I am specifically receiving this error.

     

    ==

    Caused by: com.ibm.websphere.objectgrid.ObjectGridRuntimeException: java.io.StreamCorruptedException: invalid type code: 3F 
    at com.ibm.ws.objectgrid.map.BaseMap.bytesToObject(BaseMap.java:12127) ~[wsogclient.jar:WXS8.5.0.XS [a1216.20150800]] 

    ==

     

    Would the upgrade you suggested (8.5.0.3) solve this one as well?

     

    Thanks.

     

     

     

    Updated on 2013-04-25T10:54:53Z at 2013-04-25T10:54:53Z by Dev_Dhoot
  • jhanders
    jhanders
    265 Posts

    Re: Null value getting stored in object map

    ‏2013-04-25T10:59:59Z  
    • Dev_Dhoot
    • ‏2013-04-25T10:54:28Z

    Jared,

    Thanks for the reply.

    I am specifically receiving this error.

     

    ==

    Caused by: com.ibm.websphere.objectgrid.ObjectGridRuntimeException: java.io.StreamCorruptedException: invalid type code: 3F 
    at com.ibm.ws.objectgrid.map.BaseMap.bytesToObject(BaseMap.java:12127) ~[wsogclient.jar:WXS8.5.0.XS [a1216.20150800]] 

    ==

     

    Would the upgrade you suggested (8.5.0.3) solve this one as well?

     

    Thanks.

     

     

     

    Yes.  I am the one who made the code fix.  The exception you are hitting is exactly what I would expect to see for APAR PM76020.

  • Dev_Dhoot
    Dev_Dhoot
    42 Posts

    Re: Null value getting stored in object map

    ‏2013-04-25T11:14:58Z  
    • jhanders
    • ‏2013-04-25T10:59:59Z

    Yes.  I am the one who made the code fix.  The exception you are hitting is exactly what I would expect to see for APAR PM76020.

    Thanks for the reply. I will proceed with the 8.5.0.3 level.

     

    Thanks again.

  • smart_dev
    smart_dev
    56 Posts

    Re: Null value getting stored in object map

    ‏2013-04-26T05:33:20Z  
    • jhanders
    • ‏2013-04-25T10:59:59Z

    Yes.  I am the one who made the code fix.  The exception you are hitting is exactly what I would expect to see for APAR PM76020.

    Hi Jhanders,

    Does this also hold good if I upgrade to 8.6 for WXS rather than 8.5.0.3? Please suggest.

     

    Thanks.

  • jhanders
    jhanders
    265 Posts

    Re: Null value getting stored in object map

    ‏2013-05-01T12:24:21Z  
    • smart_dev
    • ‏2013-04-26T05:33:20Z

    Hi Jhanders,

    Does this also hold good if I upgrade to 8.6 for WXS rather than 8.5.0.3? Please suggest.

     

    Thanks.

    8.6 also has the fix for this problem.