Topic
  • 4 replies
  • Latest Post - ‏2011-04-20T15:49:12Z by SystemAdmin
SystemAdmin
SystemAdmin
9224 Posts

Pinned topic ZRM problem occasional update failures and repeatable loadda

‏2011-04-19T10:11:17Z |
NOTE: this thread was migrated from projectzero.org. Some content and formatting may have been lost in the move.


Originally posted by projectzero userid: barny - f=4&t=2302#p9354
Using Smash on an IBM internal hosting (which akasman runs), appears to be zero:zero.kernel:1.1.1.4.31215, zero version gives 1.1.1.4.31129 20100601 0917

I have what seems like a bizarre problem. The problem I'm seeing is my app is running a timed update every ten minutes which reads some REST APIs and updates a ZRM resource (this is the only writes to the resource), but occasionally a zrm update fails. I dumped my data using zero model dumpdata and there is nothing obvious wrong with it. However doing a zrm reset then loaddata on the same file gave an error:

command> zero model loaddata data20110419.json
CWPZC9225I: Using default Derby database configuration. Located at -> db/resource in application root.
CWPZT0601E: Error: Command model loaddata failed
CWPZC0011E: An unhandled error was caught; message: java.lang.RuntimeException: java.lang.IllegalArgumentException: Value out of range

Which is just about as irritatingly unhelpful as an error message can be when the json file contains 70 records each 9 lines long. It could tell me what line of the json file it failed at, couldn't it?

I deleted data from the json output file of dumpdata and narrowed it down after a lot of editing, reset, loaddata cycles (most of which wouldn't be needed if the cruddy error message gave me the line number) to the this record:

[
{
"type": "userEmailMapping",
"fields": {
"myEmail": "a@b.com",
"yourEmail": "c@d.com",
"id": 103,
"updated": "2011-04-18 04:58:07.999"
}
}
]

More editing within the record and I found that I can edit the "updated" line to end .998 (or anything but .999) and it loads, change it to .999 and it doesn't. I can edit any other values into the other fields but it is the .999 value in the updated field which causes the data to load or not.

This is completely repeatable - it always gives an error when I use zrm model loaddata to load the above file, yet if I edit the updated timestamp to end in anything else but .999 it always loads.

I assume that my original problem is when an update happens to have the .999 timestamp while the app is running. The updated timestamp is automatically generated so there is no simple way to avoid this happening 1 in 1000 times.

Am I going bonkers, or doing something stupid, or is this a known problem or is it a newly discovered bug?
Updated on 2011-04-20T15:49:12Z at 2011-04-20T15:49:12Z by SystemAdmin
  • SystemAdmin
    SystemAdmin
    9224 Posts

    Re: ZRM problem occasional update failures and repeatable loadda

    ‏2011-04-19T11:26:24Z  

    Originally posted by projectzero userid: barny - f=4&t=2302#p9356
    Some more diagnostic/info: while the loaddata problem (refusing to load records with updated field ending in .999) is unchanged, I can only imagine that what is happening when my app is running is that the record is getting updated with a .999 in the "updated" field, it is when this record is updated again that the exception occurs. Or at least I can't understand how else the data exported by dumpdata includes a record with an "updated" field ending in .999.
  • SystemAdmin
    SystemAdmin
    9224 Posts

    Re: ZRM problem occasional update failures and repeatable loadda

    ‏2011-04-19T12:03:22Z  

    Originally posted by projectzero userid: barny - f=4&t=2302#p9357
    Having got stuck with some of my data with the .999 "updated" values, I thought I could get around it by dumping the data, editing it so there weren't any .999s (actually there was only one) and reloading it using loaddata. However now I'm finding that although the loaddata succeeeds many of the records in the database have an updated field ending in .999 - so my data and maybe the whole application is doomed.

    I'm close to giving up on this timer-monitoring. In desperation I've put a black magic function in there which guards the update() by delaying if the system time shows that it is more than 600ms into the second, though why I should have to do this is beyond me.
  • SystemAdmin
    SystemAdmin
    9224 Posts

    Re: ZRM problem occasional update failures and repeatable loadda

    ‏2011-04-20T14:45:43Z  

    Originally posted by projectzero userid: ngawor - f=4&t=2302#p9360
    I will take a look at this today -- if you get a chance, can you run with -d to produce the stack trace? Your model file would also be helpful.

    Thanks,
    Nell
  • SystemAdmin
    SystemAdmin
    9224 Posts

    Re: ZRM problem occasional update failures and repeatable loadda

    ‏2011-04-20T15:49:12Z  

    Originally posted by projectzero userid: ngawor - f=4&t=2302#p9361
    I have reproduced the problem -- it's definitely a bug and I have opened bug 9727. http://www.projectzero.org/bugzilla/sho ... gi?id=9727

    Thanks,
    Nell