Topic
  • 12 replies
  • Latest Post - ‏2012-07-11T17:51:37Z by SystemAdmin
johnjingle
johnjingle
12 Posts

Pinned topic Is there a programatic way to handle potential race conditions in Filenet.

‏2012-04-20T12:22:54Z |
We have an Event Handler subscribed to the CE Creation Event. The event handler modifies the source object's permissions (Adds ACE's to the ACL). The solution has a workflow that creates an object in mid-workflow, thus firing the Creation Event handler and so causing a race condition and the Event handler to fail. Can code be written to handle this situation eg somehow ordering the processing or should the handler logic be called directly by the workflow?
Updated on 2012-07-11T17:51:37Z at 2012-07-11T17:51:37Z by SystemAdmin
  • drdamour
    drdamour
    40 Posts

    Re: Is there a programatic way to handle potential race conditions in Filenet.

    ‏2012-04-20T17:29:37Z  
    I expect bill will chime in here, but my opinion on this is: no

    IMO you've found a pretty big hole in the P8 Platform here that has a couple of workarounds but no clean solution.

    CE has cooperative locking (which has timeouts) but it updates the USN, and the caller's have to agree to check for the lock (none of the OOTB stuff does). Nice thing about cooperative locking is that it has a timeout
    CE also has check out/check in which is a strong lock that is a good way to say (this is mine) for a while, but you have to clearly define who the me is, and it doesn't have timeouts (So stuff can get locked indefinitely until manual intervention which sucks).

    bill's article @ http://www.ibm.com/developerworks/data/library/techarticle/dm-0910filenetp8api/ covers some of this

    I've done code where save/update failures because of USN being the unexpected value go into a pausing loop waiting for their turn to apply their change. the trick is to make sure nothing substantial has changed on the ce object that you care about, you do this by keeping track of the old values you verified as ok to update and compare them against the latest values in the loop. If the values have changed you then have to decide if it's ok to apply your updates, or error out. It's always ugly code, and recovery can be tough.

    In the special case that you always know you don't care what the current value is, you can just null out the USN so it works, but if you ever do this, it should make you uneasy. (I end up cheating and doing this a lot do to time constraints)

    the reason you don't commonly run into this is because the References to the CE objects usually exists for less than a second something like:

    Get CE Object Ref
    Update CE Object
    Commit CE Object

    very rarely do you keep the handle to the CE object for any significant amount of time, in fact it's best to often do everything with a fetchless batch so it all happens in 1 nearly instantaneous atomic transaction. So as bill said in the other thread, it's surprising that the window of time is big enough for you to run into this.


    I am Just a new Boy,
    A Stranger in this Town,
    Where are All the Good Times,
    Who's Gonna Show this Stranger Around?
    Check out our Agile ACM Catalogue: Widgets, APIs, & Components for Building Solutions
  • SystemAdmin
    SystemAdmin
    693 Posts

    Re: Is there a programatic way to handle potential race conditions in Filenet.

    ‏2012-04-24T01:18:14Z  
    I'm not completely sure I understand the details of the scenario you are describing. Is this the scenario?

    You launch a workflow. Somewhere along the way, that running workflow creates an object XYZ. Your creation event handler updates the ACL on object XYZ. Later in your workflow, you make additional changes to XYZ, and you get an exception E_OBJECT_MODIFIED.

    If that is the scenario, it's definitely a race condition. The workflow and the event handler code are acting on the same object in parallel. I can think of a couple of approaches to resolve this.

    1. As you mentioned, do the ACL update from the workflow and eliminate the async event subscription. Nobody likes to see duplicated logic, so it's unfortunate to have the ACL update code in the handler (presumably used elsewhere). This is likely the simplest solution to implement.

    2. If you are using CE 5.1 (and you are a little bit lucky in your business requirements), you can update the ACL in a ChangePreprocessor handler before the object is actually created.

    3. If the ACL update is easy to detect, you could have the workflow wait until it sees that it has been done and only move forward after that. This requires polling the CE for changes to the object, which is a drag (both emotionally and in performance :-))

    4. You could have a CE event handler notify the (paused) workflow that the ACL update has been done.
  • SystemAdmin
    SystemAdmin
    693 Posts

    Re: Is there a programatic way to handle potential race conditions in Filenet.

    ‏2012-04-24T01:19:52Z  
    I'm not completely sure I understand the details of the scenario you are describing. Is this the scenario?

    You launch a workflow. Somewhere along the way, that running workflow creates an object XYZ. Your creation event handler updates the ACL on object XYZ. Later in your workflow, you make additional changes to XYZ, and you get an exception E_OBJECT_MODIFIED.

    If that is the scenario, it's definitely a race condition. The workflow and the event handler code are acting on the same object in parallel. I can think of a couple of approaches to resolve this.

    1. As you mentioned, do the ACL update from the workflow and eliminate the async event subscription. Nobody likes to see duplicated logic, so it's unfortunate to have the ACL update code in the handler (presumably used elsewhere). This is likely the simplest solution to implement.

    2. If you are using CE 5.1 (and you are a little bit lucky in your business requirements), you can update the ACL in a ChangePreprocessor handler before the object is actually created.

    3. If the ACL update is easy to detect, you could have the workflow wait until it sees that it has been done and only move forward after that. This requires polling the CE for changes to the object, which is a drag (both emotionally and in performance :-))

    4. You could have a CE event handler notify the (paused) workflow that the ACL update has been done.
    Sorry, I meant to say somewhere in my list that drdamour's suggestion of using cooperative locking would also work, though you don't want event handler code waiting around too long for the lock to clear.
  • johnjingle
    johnjingle
    12 Posts

    Re: Is there a programatic way to handle potential race conditions in Filenet.

    ‏2012-04-24T10:38:51Z  
    Thank agian you very much for your help. As it happens we are going to use 5.1 and I will
    implement the Change Preprocessor and see how that goes.
  • johnjingle
    johnjingle
    12 Posts

    Re: Is there a programatic way to handle potential race conditions in Filenet.

    ‏2012-04-24T15:18:35Z  
    Thank agian you very much for your help. As it happens we are going to use 5.1 and I will
    implement the Change Preprocessor and see how that goes.
    Apologies but I did not explain myself very well.
    In fact we have a User Interface that uses CMIS to create a Folder object, the UI then uses a RESTful call to run "workflow A" which performs some updating of the Folder including renaming it.

    It was the asynchronous Creation Event handler that was changing the ACL and the "workflow A" that were in a race, I believe.

    If we use a Change Preprocessor in this case described above then the synchronous Change Preprocessor will be guaranteed complete befote "workflow A"runs and no race? Or may we still et a race condition?
  • SystemAdmin
    SystemAdmin
    693 Posts

    Re: Is there a programatic way to handle potential race conditions in Filenet.

    ‏2012-04-24T16:56:47Z  
    Apologies but I did not explain myself very well.
    In fact we have a User Interface that uses CMIS to create a Folder object, the UI then uses a RESTful call to run "workflow A" which performs some updating of the Folder including renaming it.

    It was the asynchronous Creation Event handler that was changing the ACL and the "workflow A" that were in a race, I believe.

    If we use a Change Preprocessor in this case described above then the synchronous Change Preprocessor will be guaranteed complete befote "workflow A"runs and no race? Or may we still et a race condition?
    When you use a ChangePreprocessor, your changes occur synchronously and will have been performed when the creation RPC returns to you. A ChangePreprocessor is like a tiny piece of application code right at the edge of the CE server as it receives requests. Unlike sync/async event handlers, a ChangePreprocessor operates with the security context of the calling user (which shouldn't be a problem for the ACL update you are planning).

    Since you are doing it as part of creation, pay attention to the notes about availability of properties for a creation ChangePreprocessor. Is the original ACL coming from the class DefaultInstancePermissions? If so, I'm not sure whether or not the Permissions property will already be populated. If not, you could always look up the default on the class.

    Let us know how this turns out. ChangePreprocessors are a new feature in 5.1, and it's always nice to hear feedback.
  • johnjingle
    johnjingle
    12 Posts

    Re: Is there a programatic way to handle potential race conditions in Filenet.

    ‏2012-04-25T11:12:54Z  
    We have been looking at Change Preprocessors in 5.1 but feel that our safest approach
    will be with option 1 - using Component Integrator in a system step. The intended environment
    is Single Sign-On using Tivoli.
  • SystemAdmin
    SystemAdmin
    693 Posts

    Re: Is there a programatic way to handle potential race conditions in Filenet.

    ‏2012-05-10T11:05:49Z  
    When you use a ChangePreprocessor, your changes occur synchronously and will have been performed when the creation RPC returns to you. A ChangePreprocessor is like a tiny piece of application code right at the edge of the CE server as it receives requests. Unlike sync/async event handlers, a ChangePreprocessor operates with the security context of the calling user (which shouldn't be a problem for the ACL update you are planning).

    Since you are doing it as part of creation, pay attention to the notes about availability of properties for a creation ChangePreprocessor. Is the original ACL coming from the class DefaultInstancePermissions? If so, I'm not sure whether or not the Permissions property will already be populated. If not, you could always look up the default on the class.

    Let us know how this turns out. ChangePreprocessors are a new feature in 5.1, and it's always nice to hear feedback.
    > Unlike sync/async event handlers, a ChangePreprocessor operates with the security context of the calling user

    Dear WJCarpenter, could you provide some proof that synchronous event handlers are executed in the security context other than that of the calling user? This topic is not covered in documentation, but my experience shows that synchronous handler is always executed in the security context of the calling user (just like change preprocessor). What actually makes sense to me. But since you claim the opposite, it would be good to clear the things up.
  • SystemAdmin
    SystemAdmin
    693 Posts

    Re: Is there a programatic way to handle potential race conditions in Filenet.

    ‏2012-05-12T23:46:10Z  
    > Unlike sync/async event handlers, a ChangePreprocessor operates with the security context of the calling user

    Dear WJCarpenter, could you provide some proof that synchronous event handlers are executed in the security context other than that of the calling user? This topic is not covered in documentation, but my experience shows that synchronous handler is always executed in the security context of the calling user (just like change preprocessor). What actually makes sense to me. But since you claim the opposite, it would be good to clear the things up.
    5CK5 wrote:
    proof that synchronous event handlers are executed in the security context other than that of the calling user?

    I should perhaps have been a little more precise in my terminology. From J2EE's point of view, it is the calling user's security context. For access checks for CE objects, the CE server disables authorization completely when it runs a handler. The distinction I meant to make what that the CE server does not do that (disabling) when performing actions on behalf of a Change Preprocessor.
  • johnjingle
    johnjingle
    12 Posts

    Re: Is there a programatic way to handle potential race conditions in Filenet.

    ‏2012-05-16T09:24:30Z  
    Thanks. We solved the problem by mimicing CE_Operations and have added a queue and a Component Manager? If that is he correct terminolgy.
  • SystemAdmin
    SystemAdmin
    693 Posts

    Re: Is there a programatic way to handle potential race conditions in Filenet.

    ‏2012-07-11T13:45:26Z  
    5CK5 wrote:
    proof that synchronous event handlers are executed in the security context other than that of the calling user?

    I should perhaps have been a little more precise in my terminology. From J2EE's point of view, it is the calling user's security context. For access checks for CE objects, the CE server disables authorization completely when it runs a handler. The distinction I meant to make what that the CE server does not do that (disabling) when performing actions on behalf of a Change Preprocessor.
    > wjcarpenter wrote:
    > 5CK5 wrote:
    proof that synchronous event handlers are executed in the security context other than that of the calling user?
    >
    > I should perhaps have been a little more precise in my terminology. From J2EE's point of view, it is the calling user's security context. For access checks for CE objects, the CE server disables authorization completely when it runs a handler. The distinction I meant to make what that the CE server does not do that (disabling) when performing actions on behalf of a Change Preprocessor.

    Again, this contradicts my experience. You say, with event handler, CE disables authorization completely. But the following simple scenario can be easily reproduced to show that your statement is not valid:

    • let's have userA and userB
    • let's have an object where modification access is granted to userA but not userB
    • let's have another object with synchronous event handler for some event
    • in event handler there is code modifies the first object
    • then when we connect to Content Engine as userB and make a change that triggers event handler we get security permission exception, as userB is not allowed to modify the object

    This only means to me that with synchronous event handler authorization checks are not disabled. Actually, this completely makes sense to me.
  • SystemAdmin
    SystemAdmin
    693 Posts

    Re: Is there a programatic way to handle potential race conditions in Filenet.

    ‏2012-07-11T17:51:37Z  
    > wjcarpenter wrote:
    > 5CK5 wrote:
    proof that synchronous event handlers are executed in the security context other than that of the calling user?
    >
    > I should perhaps have been a little more precise in my terminology. From J2EE's point of view, it is the calling user's security context. For access checks for CE objects, the CE server disables authorization completely when it runs a handler. The distinction I meant to make what that the CE server does not do that (disabling) when performing actions on behalf of a Change Preprocessor.

    Again, this contradicts my experience. You say, with event handler, CE disables authorization completely. But the following simple scenario can be easily reproduced to show that your statement is not valid:

    • let's have userA and userB
    • let's have an object where modification access is granted to userA but not userB
    • let's have another object with synchronous event handler for some event
    • in event handler there is code modifies the first object
    • then when we connect to Content Engine as userB and make a change that triggers event handler we get security permission exception, as userB is not allowed to modify the object

    This only means to me that with synchronous event handler authorization checks are not disabled. Actually, this completely makes sense to me.
    Perhaps it is a side issue to what you are trying to describe, but you know it's not supported for a synchronous event handler to modify the triggering object, right? Attempting that is undefined territory.

    But suppose you modify your scenario so that the sync event handler modifies some other object (ie, not the triggering object). Do you still see the situation you described for userA vs userB? (If yes, please mention the specific CE server release involved. So far, that sounds to me like it would be a bug.)