Topic
15 replies Latest Post - ‏2014-10-01T14:35:29Z by jaysonshivey
JuanCarlosLopera
JuanCarlosLopera
47 Posts
ACCEPTED ANSWER

Pinned topic IBM BPM 8 Shared Object problems

‏2013-01-30T21:54:29Z |
Hi everyone,
I'm looking for help or guidance with the following issue we are having in our Process implementation.

the scenario is:

1. we have a process in which we are using the IBM BPM 8 new feature Shared Business Object.

2. we use this approach to synchronize multiple threads on shared data.

The problem is that we've noticed that the Share Business Object functionality creates a version of the BO at some points. the thing here is that this versioning approach makes that the Data Base File System reach 10 GB of storage.

my questions are:

1. can someone point me to a good source detailing the Share BO behavior.
2. is there a way to maintain just a number of version of the BO.
3. when did the Shared BO ends its life-cycle. the documentation says "as long as the originating BPD is active". does this means that when the bpd became completed the BO is removed?
4. is there a way to monitor the Share BO behavior?
5. any suggestion on solving this issue?

ill really appreciate your help

Regards
Juan
Updated on 2013-02-18T15:41:18Z at 2013-02-18T15:41:18Z by jmac_EmeriCon
  • kolban
    kolban
    3314 Posts
    ACCEPTED ANSWER

    Re: IBM BPM 8 Shared Object problems

    ‏2013-01-31T20:05:04Z  in response to JuanCarlosLopera
    Hi Juan,
    I haven't had a need yet to use the Shared Object construct yet but I'm willing to learn. Let us step back. I understand the Shared Object concept to be the notion that the same business object instance can be seen by multiple processes. This means that a change to its properties in one process will be seen as changes in another. Ok ... great. If my understanding is correct ... useful enough.

    Now, in your story, you say you are using it to "synchronize multiple threads on shared data". Can you elaborate on that from a functional perspective? What are you trying to achieve.

    Next you start to talk about "versioning" ... hmmm ...

    I don't follow that. I understand that an instance of a shared object has a "key" so that I can have multiple instances of the shared business object and each instance has a key for access. That makes sense to me ... is this what you mean by versioning?

    10 Gbytes is a lot of data :-) What makes you think there is some usage of the DB that is Shared Object related? Not saying you are wrong, just curious on the thinking.
    • SystemAdmin
      SystemAdmin
      7615 Posts
      ACCEPTED ANSWER

      Re: IBM BPM 8 Shared Object problems

      ‏2013-02-01T16:39:49Z  in response to kolban
      I haven't played with the shared objects either, but, if I understood the conversation with the designer, one of the issues you run into with multiple threads updating the same data (no matter what the technology) is how to handle someone updating based on an out of synch copy of the data. As I understood it, rather then have the shared data model possibly have data loss, somewhere in the object every version of the BO is saved, much like the time machine in the Process Designer. This would allow a developer to see that person A updated the firstName column then person B set it back to the original value. This would allow you to determine how you wanted to merge the data at whatever point you feel is appropriate.

      10GB would be a massive amount of data for this, but you can see where the data would accumulate as changes happened. Now, all of this is based off of a conversation with one of the designers, so what was acutally implemented may be different...

      Andrew Paier | Director of Special Operations | BP3 Global, Inc. www.bp-3.com
      • kolban
        kolban
        3314 Posts
        ACCEPTED ANSWER

        Re: IBM BPM 8 Shared Object problems

        ‏2013-02-02T15:31:57Z  in response to SystemAdmin
        Oooh ... now it gets interesting... I had thought that a shared variable would be "last change is current value" ... simple semantics. So whenever a BPD is awoken to perform its next step the runtime would retrieve the current and single value of the variable. The alternative design would be similar to BPEL variables which have lock access such that all access to the variable is "mutexed". Only one process could be changing the variable at a time. This is similar to the database concept of DB isolation level locks (see: http://en.wikipedia.org/wiki/Isolation_level). If the shared objects in BPM are doing something other than simple access or isolation levels ... that would be "unusual" and should definitely be exposed in the semantics of shared objects.

        Neil
        • jmac_EmeriCon
          jmac_EmeriCon
          277 Posts
          ACCEPTED ANSWER

          Re: IBM BPM 8 Shared Object problems

          ‏2013-02-02T16:40:42Z  in response to kolban
          My 2 cents.

          The doc on Shared Objects is incredibly thin. I have a PMR (94281,756,000 probably not visible to anyone outside of IBM) open for about a month requesting more doc on them, and detailing my issues with them, but I have heard nothing yet that is helpful to this or the other thread on Shared Objects.

          There are definitely versions stored in the database, but it seems to me that you can only get at the most current one legally (i.e. no DB read). There is a load() method that is mentioned that I can find no doc on anywhere.

          This method might be the key to accessing the multiple versions but at this point I don't have a clue what it does. This is the subject of my PMR.

          I am thinking that shared objects are an idea to solve some issues, but at the moment they are not fully up to snuff. Currently other than playing around with them, I would likely avoid them in any production solutions. Please note that I am always accused of being overly cautious, so I am not saying that shared objects are broken, I am simply wanting to see more doc on them, since the research I have done so far has left me with many questions.

          Regards

          John


          _______________________________________________________________________

          John McDonald

          EmeriCon, LLC
          • scheu
            scheu
            4 Posts
            ACCEPTED ANSWER

            Re: IBM BPM 8 Shared Object problems

            ‏2013-02-15T17:09:33Z  in response to jmac_EmeriCon
            Greetings John,

            Each shared business object has a "key" which identifies the object and a "version" which is used internally to disambiguate copies of the data.

            Scenario:

            So let's say that you have a shared business object Employee that has fields name, role, office.

            Assume the Employee object has the following state:
            name = John Smith
            role = entry
            office = 345
            key = abc123
            version = 1

            This object is passed as an argument to two activities running in parallel.
            Each activity has a copy of the original state.

            Activity 1 promotes this employee to management. This copy will have the following information:

            name = John Smith
            role = manager
            office = 345
            key = abc123
            version = 1
            dirty=true

            Note that the version did not change, but the dirty flag was set.

            Activity 2 assigns a new office to the employee. This copy will have the following information:

            name = John Smith
            role = entry
            office = executive suite 111
            key = abc123
            version = 1
            dirty = true

            At the end of the activity or an invoke of save() will cause the data to be persisted. Let's assume in this case that Activity 1 completes first. The object will be changed to this:

            name = John Smith
            role = manager
            office = 345
            key = abc123
            version = 2
            dirty=false

            When Activity 2 completes the data is persisted. The runtime will notice that Activity 2's copy is stale and will only update the changed fields.
            name = John Smith
            role = manager
            office = executive suite 111
            key = abc123
            version = 3
            dirty = false
            The load() command will simply load the current state of the data. The key paramegter to the load command is used if you want to load a different shared data instance.

            Rich Scheuerle
            IBM BPM
            • jmac_EmeriCon
              jmac_EmeriCon
              277 Posts
              ACCEPTED ANSWER

              Re: IBM BPM 8 Shared Object problems

              ‏2013-02-15T19:54:27Z  in response to scheu
              Rich:

              Thanks for the response... a couple of questions

              Where is this documented?

              You seem to be implying that the actual object is passed to the parallel activities, and not just the key to the object I just want to be certain that you are saying that this is the proper way to use these shared objects.

              Where would I see the dirty flag, it does not show up in the attributes when I log a shared object. What I see is:
              
              <variable type=
              "TestBO1" sharedKey=
              "SharedObject.f03abd51-c5e5-40c3-80c7-ec39cfa0bd89" sharedVersionID=
              "2"> <field1 type=
              "String"><![CDATA[Initial value]]></field1> <field2 type=
              "Integer"><![CDATA[666]]></field2> <field3 type=
              "Boolean"><![CDATA[
              
              true]]></field3> <field4 type=
              "Decimal"><![CDATA[12.34]]></field4> </variable>
              


              I recall getting an error using the load method but that was in 800, I will try this again later today in 801.

              Thanks

              John


              _______________________________________________________________________

              John McDonald

              EmeriCon, LLC
            • jmac_EmeriCon
              jmac_EmeriCon
              277 Posts
              ACCEPTED ANSWER

              Re: IBM BPM 8 Shared Object problems

              ‏2013-02-15T21:29:25Z  in response to scheu
              Rich:

              Further Information for you.

              I am attempting to build a Shared Business Object Factory as described here in the 8th bullet under number 6.

              The load method appears to be unusable in this situation.

              As I said before, the doc available (or at least that I can find) is woefully inadequate.

              Thanks for your help

              John

              _______________________________________________________________________

              John McDonald

              EmeriCon, LLC
              • scheu
                scheu
                4 Posts
                ACCEPTED ANSWER

                Re: IBM BPM 8 Shared Object problems

                ‏2013-02-18T15:17:03Z  in response to jmac_EmeriCon
                >You seem to be implying that the actual object is passed to the parallel activities, and not just the key to the object I just want to be certain that you are saying that >this is the proper way to use these shared objects.

                The best practice is to pass the shared object to each parallel activity. Each activity has a local copy of the shared object, but that copy is updated to the current
                version from the data store when (a) execution starts (b) execution ends or (c) the load() method is used.
                See http://pic.dhe.ibm.com/infocenter/dmndhelp/v8r0m1/topic/com.ibm.wbpm.bpc.doc/modeling/topic/declaring_variables_A.html?resultof=%22%73%68%61%72%65%64%22%20%22%73%68%61%72%65%22%20%22%6f%62%6a%65%63%74%22%20 and http://pic.dhe.ibm.com/infocenter/dmndhelp/v8r0m1/topic/com.ibm.wbpm.wle.editor.doc/modeling/topic/cdev_jsapi_declaringandpassingvariables.html?resultof=%22%73%68%61%72%65%64%22%20%22%73%68%61%72%65%22%20%22%6f%62%6a%65%63%74%22%20

                >Where would I see the dirty flag, it does not show up in the attributes when I log a shared object.

                It is likely that the dirty flag is only serialized when set.
                In code you can access the dirtyFlag with the isDirty() method.
                >I am attempting to build a Shared Business Object Factory as described here in the 8th bullet under number 6.
                >The load method appears to be unusable in this situation.

                I am not sure what you are attempting to do. You must first create a business object prior to loading it. If you believe that you have found a problem with the product, please add the specific scenario to the PMR that you opened.
                I recall getting an error using the load method but that was in 800, I will try this again later today in 801.
                • jmac_EmeriCon
                  jmac_EmeriCon
                  277 Posts
                  ACCEPTED ANSWER

                  Re: IBM BPM 8 Shared Object problems

                  ‏2013-02-18T15:41:18Z  in response to scheu
                  Rich:

                  I am not trying to be flip here, but you have given me 3 times more information than I have received from the PMR. The exact scenario of what I am attempting and my request for doc is written in the PMR.

                  I am tied up this morning but will investigate your links within the next day or so.

                  Thanks for the help

                  John


                  _______________________________________________________________________

                  John McDonald

                  EmeriCon, LLC
        • SystemAdmin
          SystemAdmin
          7615 Posts
          ACCEPTED ANSWER

          Re: IBM BPM 8 Shared Object problems

          ‏2013-02-04T14:59:09Z  in response to kolban
          Neil -

          In general I think the shared object is useing the "last change is current value". The big question is, if you and I both update a field in the object close enough in time that we don't see each other's changes, what happens. My understanding is that both "version" of the data save are kept in the object's history. That way, in theory, a developer could detect that state and decide what to do with it. Otherwise there is potentially data loss.

          As is mentioned by John, I think we are still in very early days with the shared object, so much of its potential has yet to be unleased or even documented. Hopefully that will change over time.

          Andrew Paier | Director of Special Operations | BP3 Global, Inc. www.bp-3.com
  • JuanCarlosLopera
    JuanCarlosLopera
    47 Posts
    ACCEPTED ANSWER

    Re: IBM BPM 8 Shared Object problems

    ‏2013-02-04T16:19:45Z  in response to JuanCarlosLopera
    Hi everyone,

    Sorry for taken so long to reply to the post.

    First of all i wanna reply to Neil's questions and then give a little background on the issue.
    "Now, in your story, you say you are using it to "synchronize multiple threads on shared data". Can you elaborate on that from a functional perspective? What are you trying to achieve."

    ANSWER: this is the scenario. we are using a BO that contains a list of complex objects that must be processed by a sub process each one. so we use the multi instance loop to accomplish this. inside the sub process these independent flow threads also update the container object which is common to all opened threads. Here we use the Share Object concept to allow multiple updates in the same share object reference data.
    "Next you start to talk about "versioning" ... hmmm ...

    I don't follow that. I understand that an instance of a shared object has a "key" so that I can have multiple instances of the shared business object and each instance has a key for access. That makes sense to me ... is this what you mean by versioning?"

    ANSWER: The behavior of the share business object that we are seeing is that every modification on the object data creates a new version of the object so each point where you sets a new value to the Share BO it create a new version. opposed as we expect this is not a share memory reference to the same object. as other says in the forum the share object resolves itself by versioning from each updates and merging its data.

    "10 Gbytes is a lot of data :-) What makes you think there is some usage of the DB that is Shared Object related? Not saying you are wrong, just curious on the thinking."

    ANSWER: yes Neil, that a lot of data considering that our process updates the share BO at least one time at each activity. The platform administrator says to us that there process instances with hundreds of versions of share objects. the development environment collapse the last week due to File System store limit reach.

    The thing here is that the reduced docs about this feature says that this is a way to solve our scenario and talks about an object not passed by copy between engines but an object passed by reference which lead to think about shared memory portions not multiple object copies. our object size its about 58 kb and so far there are 10000 instance executed in our environment.
    Hope this info makes the issue clear.

    PD: we are about 1 month to go into production.
    • SystemAdmin
      SystemAdmin
      7615 Posts
      ACCEPTED ANSWER

      Re: IBM BPM 8 Shared Object problems

      ‏2013-02-04T18:47:22Z  in response to JuanCarlosLopera
      Juan -

      In your scenario, is each thread of the MIL working on its own sub memeber of the wrapping BO, or are there times when 1 thread will be updating data that another thread needs to read/write? If you aren't doing the 2nd, then it is likely that Share Business Object was really more heavy weight than you want.

      Just for the purposes of discussion, lets say we have a BO called "WorkItem" it has 3 members "workToBeDone" (string), "whenToDoIt" (Date), "completedByUser"(String).

      Now lets say we have another BO "WorkRequestList" it has a "requestor" (String) and "stuffRequested" (list of WorkItem).

      So our requestor comes in, requests a bunch of stuff, and we want to use a MIL to allow all those things to be done in parallel. Now, each member of the MIL is only going to alter its single workItem, updating it to have the completed by User. Clearly this update could be much more complex but the point is that the update is limited to just that scope. Since that is true, then there really is no need to coordinate between the different threads. I would just make the MIL take 2 inputs and give back a single output. The inputs would be the WorkRequestList, and the step number so they no which of the items they are addressing.

      I'm assuming here that we want to hand in the parent object becasue that has some overall information on it they need to know and is shared between all the items.

      The output would be a single WorkItem, and would just over-write the specific entry the MIL step was addressing.

      Now if any 2 items in the MIL are updating the same WorkItem, or if any of them are updating the stuff in the WorkRequestList, this pattern will not work.

      Andrew Paier | Director of Special Operations | BP3 Global, Inc. www.bp-3.com
      • heyScozzaro
        heyScozzaro
        54 Posts
        ACCEPTED ANSWER

        Re: IBM BPM 8 Shared Object problems

        ‏2013-09-19T10:03:29Z  in response to SystemAdmin

        Hello Andrew

        I am having a similar problem with a shared object and auto-save. I'll describe a simplified scenario: I have a process where I and my collegue can both update a shared object, concurrently, through two different activities.

        I have read only access to the data, and it's tied to my shared object's sub-objects, while my collegue can edit the data. If I keep my coach open, and he updates the data, the following occurs:

        - if I just close my browser's tab, and then reopen my activity, then I see the correctly updated data.

        - if I take any action, as clicking a button, the coach view automatically saves the data on the shared object, thus overwriting what was already updated by my collegue.

        Is there a way to prevent such autosaving from the coach-view, programmatically?

        Thank you

        Tony

        • Bruno.Cruz
          Bruno.Cruz
          3 Posts
          ACCEPTED ANSWER

          Re: IBM BPM 8 Shared Object problems

          ‏2013-12-09T13:43:30Z  in response to heyScozzaro

          Hello all,

           

          I receive an error when i use Shared Objects. When i have concurrent access on the flow, in some moments i receive the following error:

           

          org.springframework.dao.DataIntegrityViolationException: PreparedStatementCallback; SQL [insert into BPM_SHARED_OBJECT_INSTANCE (INSTANCE_VERSION_ID,INSTANCE_VERSION,JDATA,CREATED_ON,DEFINITION_VERSION_ID,CREATED_BY_USER_ID) values (?,?,?,?,?,?)]; ORA-00001: restrição exclusiva (PROCESSD.BPM_SHARED_OBJECT_INSTANCE_UQ) violada ; nested exception is java.sql.SQLIntegrityConstraintViolationException: ORA-00001: restrição exclusiva (PROCESSD.BPM_SHARED_OBJECT_INSTANCE_UQ) violada

           

          Somebody can help-me?

          Thanks!

          Bruno Cruz

           

          • jaysonshivey
            jaysonshivey
            3 Posts
            ACCEPTED ANSWER

            Re: IBM BPM 8 Shared Object problems

            ‏2014-10-01T14:35:29Z  in response to Bruno.Cruz

            Hi Bruno,

            Did you find a resolution for this issue?