Topic
  • 9 replies
  • Latest Post - ‏2013-10-16T11:38:28Z by ShahMihir
smart_dev
smart_dev
56 Posts

Pinned topic com.ibm.websphere.objectgrid.ObjectGridRuntimeException: XDF Error: Fields still need serializers

‏2013-10-08T12:56:59Z |

Hi,

We are working on WXS 8.6.0.3 stand-alone remote topology and having clients installed on WAS7.0.0.29. We are using WXS API to connect with the grid and off load the cache into the remote grid.

We are seeing below exceptions a lot recently although not hampering the app functionality at this point. Can somebody please help understanding what is going wrong with Field type:java.util.Set not being serialized?

===========

Caused by: com.ibm.websphere.objectgrid.ObjectGridRuntimeException: XDF Error: Fields still need serializers : Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994

at com.ibm.ws.objectgrid.plugins.io.dataobject.values.ValueDataImpl.getObject(ValueDataImpl.java:336) ~[wsogclient.jar:WXS8.6.X.XS [cf31329.18064852]]
at com.ibm.ws.objectgrid.DiffMap.get(DiffMap.java:1058) ~[wsogclient.jar:WXS8.6.X.XS [cf31329.18064852]]
at com.ibm.ws.objectgrid.ObjectMapImpl.get(ObjectMapImpl.java:451) ~[wsogclient.jar:WXS8.6.X.XS [cf31329.18064852]]
... 127 common frames omitted
Caused by: com.ibm.ws.objectgrid.MissingSerializationInfoException: XDF Error: Fields still need serializers : Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994Field Name:logicalPortIds; Field type:java.util.Set; Field key:typeId=102 ;Domain Hash=777432994
at com.ibm.ws.objectgrid.xdf.SerializerFactory.generateDescriptor(SerializerFactory.java:615) ~[wsogclient.jar:WXS8.6.X.XS [cf31329.18064852]]
at com.ibm.ws.objectgrid.xdf.SerializerFactory.getDescriptorFromID(SerializerFactory.java:557) ~[wsogclient.jar:WXS8.6.X.XS [cf31329.18064852]]
at com.ibm.ws.objectgrid.xdf.serializers.collections.HashMapSerializer.deserializeObject(HashMapSerializer.java:320) ~[wsogclient.jar:WXS8.6.X.XS [cf31329.18064852]]
at com.ibm.ws.objectgrid.xdf.serializers.collections.HashMapSerializer.deserializeFieldToObject(HashMapSerializer.java:334) ~[wsogclient.jar:WXS8.6.X.XS [cf31329.18064852]]
at com.ibm.ws.objectgrid.xdf.serializers.GenericClassSerializer.deserializeFieldsToObject(GenericClassSerializer.java:251) ~[wsogclient.jar:WXS8.6.X.XS [cf31329.18064852]]
at com.ibm.ws.objectgrid.xdf.serializers.GenericClassSerializer.deserializeObject(GenericClassSerializer.java:194) ~[wsogclient.jar:WXS8.6.X.XS [cf31329.18064852]]
at com.ibm.ws.objectgrid.xdf.serializers.CheckTypeSerializer.deserializeObject(CheckTypeSerializer.java:87) ~[wsogclient.jar:WXS8.6.X.XS [cf31329.18064852]]
at com.ibm.ws.objectgrid.xdf.XDFSerializerPlugin.inflateDataObject(XDFSerializerPlugin.java:319) ~[wsogclient.jar:WXS8.6.X.XS [cf31329.18064852]]
at com.ibm.ws.objectgrid.plugins.io.dataobject.values.ValueDataImpl.getObject(ValueDataImpl.java:328) ~[wsogclient.jar:WXS8.6.X.XS [cf31329.18064852]]
... 129 common frames omitted
 
  • JP_Parkin
    JP_Parkin
    7 Posts

    Re: com.ibm.websphere.objectgrid.ObjectGridRuntimeException: XDF Error: Fields still need serializers

    ‏2013-10-08T18:33:57Z  

    This looks like a bug in the 8603 code - please open a PMR for this issue.  If you have any FFDC files generated at the time of the failure, please include them when opening the PMR.

    Best regards,
    JP

  • smart_dev
    smart_dev
    56 Posts

    Re: com.ibm.websphere.objectgrid.ObjectGridRuntimeException: XDF Error: Fields still need serializers

    ‏2013-10-09T07:01:31Z  

    Thanks JP_Parkin for your reply. Forgot to mention that we are using WXS API and since WXS 8.6 and all objects are java serialized in the code rather than XDF. Does that mean XDF is not set for the application because of that?

    I guess this is a side-effect due to use of java serialization when XDF is enabled by default with WXS 8.6.0.3.

     

     

  • jambo-ray
    jambo-ray
    2 Posts

    Re: com.ibm.websphere.objectgrid.ObjectGridRuntimeException: XDF Error: Fields still need serializers

    ‏2013-10-09T12:26:17Z  
    • smart_dev
    • ‏2013-10-09T07:01:31Z

    Thanks JP_Parkin for your reply. Forgot to mention that we are using WXS API and since WXS 8.6 and all objects are java serialized in the code rather than XDF. Does that mean XDF is not set for the application because of that?

    I guess this is a side-effect due to use of java serialization when XDF is enabled by default with WXS 8.6.0.3.

     

     

    I am thinking that "WXS" stands for "Websphere eXtreme Scale", but what is "XDF"?

     

  • JP_Parkin
    JP_Parkin
    7 Posts

    Re: com.ibm.websphere.objectgrid.ObjectGridRuntimeException: XDF Error: Fields still need serializers

    ‏2013-10-09T13:43:56Z  
    • smart_dev
    • ‏2013-10-09T07:01:31Z

    Thanks JP_Parkin for your reply. Forgot to mention that we are using WXS API and since WXS 8.6 and all objects are java serialized in the code rather than XDF. Does that mean XDF is not set for the application because of that?

    I guess this is a side-effect due to use of java serialization when XDF is enabled by default with WXS 8.6.0.3.

     

     

    @smart_dev - in 8.6 when using XIO + copyMode=COPY_TO_BYTES you are using XDF even in situations where your application controls the serialization of objects by implementing writeObject / readObject methods or implementing the Externalizable interface.  The only exception to this would be if you implement your own DataSerializer for the grid - in which case that will bypass the XDF serialization.

    Did you find any FFDC files to help with the identification of the error?

    @jambo-ray - XDF is the new serialization format ( eXtreme Data Format ) that is used in WXS 8.6 when running with the XIO communication protocol and using copyMode=COPY_TO_BYTES in the objectgrid.xml definition.  This serialization format is common for the Java and C# platforms so that enterprises are able to share data within the grid ( see http://pic.dhe.ibm.com/infocenter/wxsinfo/v8r6/topic/com.ibm.websphere.extremescale.doc/txsconfigxdf.html?resultof=%22%58%44%46%22%20%22%78%64%66%22%20 for more details about XDF )

  • smart_dev
    smart_dev
    56 Posts

    Re: com.ibm.websphere.objectgrid.ObjectGridRuntimeException: XDF Error: Fields still need serializers

    ‏2013-10-09T14:21:33Z  
    • JP_Parkin
    • ‏2013-10-09T13:43:56Z

    @smart_dev - in 8.6 when using XIO + copyMode=COPY_TO_BYTES you are using XDF even in situations where your application controls the serialization of objects by implementing writeObject / readObject methods or implementing the Externalizable interface.  The only exception to this would be if you implement your own DataSerializer for the grid - in which case that will bypass the XDF serialization.

    Did you find any FFDC files to help with the identification of the error?

    @jambo-ray - XDF is the new serialization format ( eXtreme Data Format ) that is used in WXS 8.6 when running with the XIO communication protocol and using copyMode=COPY_TO_BYTES in the objectgrid.xml definition.  This serialization format is common for the Java and C# platforms so that enterprises are able to share data within the grid ( see http://pic.dhe.ibm.com/infocenter/wxsinfo/v8r6/topic/com.ibm.websphere.extremescale.doc/txsconfigxdf.html?resultof=%22%58%44%46%22%20%22%78%64%66%22%20 for more details about XDF )

    JP_Parkin,

    I am referring section 2.8.2 IBM eXtreme Data Format from the recently published "WebSphere eXtreme Scale V8.6 Key Concepts and Usage Scenarios". 

    It says

    "XDF is not set when a map is configured with an ObjectGridTransformer or DataSerializer. It is not used if the Java class implements the externalizable interface or implements the serializable interface and the readObject and writeObject methods."

    What I understood from the above statement is that XDF is not used when application is implementing serializable interface which is the case with me here.

    Please correct me if I am wrong.

     

  • JP_Parkin
    JP_Parkin
    7 Posts

    Re: com.ibm.websphere.objectgrid.ObjectGridRuntimeException: XDF Error: Fields still need serializers

    ‏2013-10-09T15:01:17Z  
    • smart_dev
    • ‏2013-10-09T14:21:33Z

    JP_Parkin,

    I am referring section 2.8.2 IBM eXtreme Data Format from the recently published "WebSphere eXtreme Scale V8.6 Key Concepts and Usage Scenarios". 

    It says

    "XDF is not set when a map is configured with an ObjectGridTransformer or DataSerializer. It is not used if the Java class implements the externalizable interface or implements the serializable interface and the readObject and writeObject methods."

    What I understood from the above statement is that XDF is not used when application is implementing serializable interface which is the case with me here.

    Please correct me if I am wrong.

     

    I see where you are coming from - that statement is not entirely correct.  In this situation XDF is still used, but the control over the serialization is delegated by XDF to the writeObject / readObject processing and the XDF serialization format for a given class was not used.  Therefore, that particular class could not be shared with C# for example since we don't necessarily have the details necessary the deserialize the object without the class' readObject or readExternal methods.

    However, XDF is still involved in the serialization in order to handle a situation where you have a class that is not java Serializable that includes a reference to a class that is java Serializable.  So you have the control to limit what is serialized, but the XDF serialization infrastructure is still used - which is likely what is contributing to the issue that you are seeing above.

    Hopefully that clarifies my earlier comments and the comments in the documentation.

  • smart_dev
    smart_dev
    56 Posts

    Re: com.ibm.websphere.objectgrid.ObjectGridRuntimeException: XDF Error: Fields still need serializers

    ‏2013-10-11T06:04:33Z  
    • JP_Parkin
    • ‏2013-10-09T15:01:17Z

    I see where you are coming from - that statement is not entirely correct.  In this situation XDF is still used, but the control over the serialization is delegated by XDF to the writeObject / readObject processing and the XDF serialization format for a given class was not used.  Therefore, that particular class could not be shared with C# for example since we don't necessarily have the details necessary the deserialize the object without the class' readObject or readExternal methods.

    However, XDF is still involved in the serialization in order to handle a situation where you have a class that is not java Serializable that includes a reference to a class that is java Serializable.  So you have the control to limit what is serialized, but the XDF serialization infrastructure is still used - which is likely what is contributing to the issue that you are seeing above.

    Hopefully that clarifies my earlier comments and the comments in the documentation.

    Hi John,

    Thanks for making it clear to me. Now I understood the whole concepts behind XDF and where it is involved.

    I will try to catch some FFDC and open a PMR to get into the root cause of my problem. By the way would it also require FFDC from server side or just the client side logs will do?

    Thanks,

    Mihir

  • JP_Parkin
    JP_Parkin
    7 Posts

    Re: com.ibm.websphere.objectgrid.ObjectGridRuntimeException: XDF Error: Fields still need serializers

    ‏2013-10-11T13:21:37Z  
    • smart_dev
    • ‏2013-10-11T06:04:33Z

    Hi John,

    Thanks for making it clear to me. Now I understood the whole concepts behind XDF and where it is involved.

    I will try to catch some FFDC and open a PMR to get into the root cause of my problem. By the way would it also require FFDC from server side or just the client side logs will do?

    Thanks,

    Mihir

    Since your application is running in WAS, then typically the FFDCs will be generated in the application's log location those should be sufficient.  Send all the logs and FFDC files from that location - you should see the Exception in many of the files to confirm that you have the correct set of FFDCs.

    Best regards, JP

  • ShahMihir
    ShahMihir
    1 Post

    Re: com.ibm.websphere.objectgrid.ObjectGridRuntimeException: XDF Error: Fields still need serializers

    ‏2013-10-16T11:38:28Z  
    • JP_Parkin
    • ‏2013-10-11T13:21:37Z

    Since your application is running in WAS, then typically the FFDCs will be generated in the application's log location those should be sufficient.  Send all the logs and FFDC files from that location - you should see the Exception in many of the files to confirm that you have the correct set of FFDCs.

    Best regards, JP

    Thanks John,

    I have opened a PMR for this issue and working with IBM folks to reach the root cause. I will update the post once I receive feedback from IBM.

    Thanks,

    Mihir