Topic
  • 2 replies
  • Latest Post - ‏2013-08-28T15:19:18Z by wiedela
wiedela
wiedela
19 Posts

Pinned topic Google Protocol messag size increases on update

‏2013-03-14T15:42:43Z |
We are storing Google Protocol messages into our grids, using the serializer objects provided by the IBM Google Protocol sample. Some of our objects have lists (ie. 'repeated' fields) defined within. The problem we are having is that if we update the list in any way and try to update the Google object within the grid, the list continues to append with multiple values.

For example, if we have a list of 2 and we update the second item in the list, when we put the object back into the grid, the list has 4 items. The next update, 8 items, and so on. What we have resorted to doing retrieving the object, rebuilding a new one with the update, removing the object from the grid, and adding the newly created object. We really don't think this is the best process.

Has anyone else seen this issue? This has been extremely non-intuitive, so we are a little stuck right now.

Thanks
  • jhanders
    jhanders
    261 Posts

    Re: Google Protocol messag size increases on update

    ‏2013-06-11T18:04:50Z  

    I took a look at the sample and the sample uses Google Protocol's mergeFrom function.  In Google Protocol repeated types are analogous to Lists or growable arrays.  So for the mergeFrom function, the logic does a concatenation for repeated types instead of a true merge where you would remove duplicates.  This behavior is consistent with a List or array which can contain duplicate entries.  If Google Protocol had a concept of a repeated type that was a set then you would get the merge function that you are looking for.  The Google Protocol serializer sample showing how you can do merge; it is not meant to be production code.  For a Serializer that would be used in a production environment the hard work of merging repeated types would need to be done so as not to cause a memory leak for update operations.

    If you do not need your Serializer to be Mergeable you can have update always do a replace by updating the merge method to always keep the new one or by taking Mergeable out of the inheritance of your value serializer.

    I hope that helps answer your question.

    Jared Anderson

  • wiedela
    wiedela
    19 Posts

    Re: Google Protocol messag size increases on update

    ‏2013-08-28T15:19:18Z  
    • jhanders
    • ‏2013-06-11T18:04:50Z

    I took a look at the sample and the sample uses Google Protocol's mergeFrom function.  In Google Protocol repeated types are analogous to Lists or growable arrays.  So for the mergeFrom function, the logic does a concatenation for repeated types instead of a true merge where you would remove duplicates.  This behavior is consistent with a List or array which can contain duplicate entries.  If Google Protocol had a concept of a repeated type that was a set then you would get the merge function that you are looking for.  The Google Protocol serializer sample showing how you can do merge; it is not meant to be production code.  For a Serializer that would be used in a production environment the hard work of merging repeated types would need to be done so as not to cause a memory leak for update operations.

    If you do not need your Serializer to be Mergeable you can have update always do a replace by updating the merge method to always keep the new one or by taking Mergeable out of the inheritance of your value serializer.

    I hope that helps answer your question.

    Jared Anderson

    Jared,

    I think you nailed the answer. So, my follow-up question is, is there a Google Protocol serializer library out there that is ready for production code? If not, my team is going to need to modify the sampler program, so any guidance on that would also be greatly apprecated.

    Thanks

    Adam