5 replies Latest Post - ‏2014-06-09T21:27:48Z by AndrewPaier
57 Posts

Pinned topic Task execution verification and timer variable changes

‏2013-07-10T00:59:43Z |

Hello forum and Neil!!

We're working on a logistics business process application for a company that handles all the processing on land for maritime shipping vessels (trade, cargo, import, export).

Each vessel arrival is a complex process that encompasses quite a few sub-processes, and over 400 activities.
The most important objective in this application is that users do not forget the order and flow of the activities to be executed for an instance when a vessel arrives at port.

The second most important objective is to execute certain specific activities at an specific date/hour based on the ETA (Estimated Time of Arrival) of the vessel at port. For example, a report has to be sent to the competent authorities ETA -4hrs for every instance lauched.

To achieve this goals we have a first level process that right after providing the identification information of the instance branches out into two different processes. One (operational), that presents the activities and flow in the expected order and with the expected due dates (ETA -x hrs) and a second one (monitoring) that is based on timers linked to the established ETA.(illustration attached)

Challenge 1: Check in one process if another task has been finished in another process.
If one of this very important tasks has not occurred at an specific hour (ETA -x hrs) we need to be able to verify that is has not been executed and proceed with an escalation. Now, this is not a simple escalation in place as the timing is not measurable based on the task duration but measured on if the specific task has finished before the ETA -x hrs. This is why we chose to create a "monitoring" process…
Our challenge here is that we've been unable to notify the monitoring process that an specific task has been executed using a post-assignment task to send a variable to the main process and then into the monitoring process. (The sub-process does not necessarily ends)
Our guess is that using a UCA would be the option trigger this notification, but we've been unsuccessful on doing so. Any idea how to tackle it?

Challenge 2: Ad-Hoc change of ETA variable is not affecting already running timers.
Now, vessels are not always on time, and sometime ETA, TA, ETD, TD change and, therefore, all the due dates for the vast majority of tasks in the instance change.
The monitoring process (illustration attached) is based on timers as "gateways" that open at ETA -x hrs allowing the verification/escalation check tasks to take place. We've created an ad-hoc activity that allows the manager to change the ETA for a vessel but somehow the timers that have already started do not "honor" the new variable and instead keep running with the "old" data.
Any ideas how to solve this?

As always thanks so much for your help and time!!


Nicolas E.


    307 Posts

    Re: Task execution verification and timer variable changes

    ‏2013-07-11T17:30:33Z  in response to F7QD_Nicolas_Echavarria

    Sound like a very interesting and challenging process to model. I have no silver bullets for you, but let me tell you how I would model it and some of my thoughts.

    First, although I absolutely understand why you came up with this idea of a monitoring process, I don't like that idea. The problem is that you end up making your design brittle: the monitoring process has to understand the implementation details of your operational process or it can't understand or affect what is going on in the operational side. Thus even a simple change like a process designer changing the name of an activity can cause the whole system to fail. Another problem is that you can no longer look at the operational process and really understand what is going on. There will be "magic" that happens if the monitoring process intervenes and that makes it harder to understand what happens in exception cases like when the ETA changes. This monitoring BPD approach would be my absolute last resort if I could think of no other way. Most times, however, you should be able to get the same results with events and timers.

    This solves some of your problems with "monitoring one process from other", but let me come back to that later.

    Let me suggest two alternatives.


    The first is instead of a countdown process rather than a monitoring process. This would work well if there are several important times you have to monitor. If you have a bunch of things that need to be done at T-4h, others that need to be done by T-3h, some that can't start until T-2h, lots at T-30m, and lots at arrival, then you are golden. If there are 200 different trigger times, this becomes trickier.

    In many ways this countdown might look like your monitoring process. It has a long string of activities representing those trigger times. Each would have a timer event that listens for that amount of time to expire, but also an IME that listens for a "ETA changed" event. If that event is received it essentially just pops off the activity and re-enters the activity with the new ETA. If the time has been moved forward enough that the time has been reached the timer event would immediately fire and proceed to the next activity. (Which also might fire immediately as well if the ETA had been moved forward enough.) In between each activity you have a "fire event" activity for that timer period. The net of all of this would be a BPD that would fire an "ETA-4H" event four hours before the arrival, an "ETA-3H" three hours before, and so on. It would translate the advancement of time into a series of actual meaningful business events. Even better if it receives a time changed event you don't have to worry about changing the timers already created because the IME's will recreate everything with the new times. And best of all you don't have to worry about duplicating time evens. e.g if the ETA is 2pm and we've already fired the T-2h event, and the time is pushed back to 4pm we don't have to worry about sending the T-2H event again. (unless you want to fire again, you could easily modify this so that it did that as well.)

    Which is great. Because now instead of an opaque monitoring process that breaks encapsulation and has lots of embedded business logic you now just have a BPD that emits these meaningful events that you can listen for. But we do run into one snag.

    THE "Don't cancel things that are already started snag"

    This is a use case that isn't super easy to model and I think is one of your core challenges. The easy cases such as "escalate this if T-3h is reached" or "start this when T-3h" is reached are easy to model with IMEs. But, as I understand you, you also have a use case that is "if this has been started at T-3h, do nothing (or perhaps do something like send an email), but if it hasn't even been started we need to escalate. If this were purely fixed times, there is an out of the box feature to handle this. But since we have to accomodate the adjustments in ETA and using events it is no longer just a configuration setting and we need to adjust our model.

    Instead of just a single activity we have two parallel activities. Let me use a concrete example to make the language easier. Let us say that a "Submit Arrival Report" must be started by T-3H and must be completed/escalated by T-2H. If this was a simple timer we could do this with the "tolerance" feature. But in our event-based model we actually want to create two activities, with a split right before them and a join right after them. Let's call them "Submit Arrival Report" and "Monitor Arrival Report". (Having a monitor activity isn't as bad as a monitor BPD in my opinion since the logic is right next to each other and therefore not opaque.)

    "Submit Arrival Report" is straightforward. It has the screens needed, an IME that listens to T-2H that escalates it as needed, and an IME that listens for "Arrival Report Escalated". It also has one little trick:. the first step of the activity is firing a "arrival report started" UCA.

    "Monitor Arrival Report" is an empty implementation. (It could be a multiple event listing gateway, but I prefer to use activities so that a task is created that I can observe.) It has two listeners: an IME for "arrival report started" and an IME for "T-2H". The line leading out of the "T-2H" IME triggers an "arrival report escalated" UCA before reaching the join.

    The net of which is that in the normal case the user starts the "monitor arrival report" activity. The UCA fires and immediately removes the "monitor arrival report" task. When the user completes the report the split is reached by both tokens and things proceed as normal.

    If, instead, the T-3H is reached before the activity is started, the monitor will catch it and fire the "escalate" event. (Which you can handle as needed.)

    If, the user has started (but not finished) the task when T-3H is received, nothing happens because the monitor activity has been cleared by the "approval report started" UCA. But if it is still not finished when T-2H is received, then the task will escalated as expected.

    Side note: make sure you either use complex gateways for the joins, or be careful not to leave stranded tokens.

    The upside of all of this is that even though there is a lot of events going on here, they are all business meaningful. It's relatively easy to look at that process and say "this is what happens when we get two hours from arrival", "this is what happens at T-2h".


    More or less you use the same trick as before in approaching the events, but instead of listening for the T-3H, T-2H,  and so forth the monitoring activity listen for the "ETA change" events directly. This is somewhat harder because you can't have any listeners on your non-monitoring activities (because they'd cancel anytime an ETA changed). So you'd have to essentially have two monitors activities for every one "do stuff" activity. The first monitor looking for "must be started" and the second monitor looking for "must be finished".

    It's also not as clean because each of the monitors has a lot more work to do looking at date/times and calculating if anything needs to be done. Not to mention it would be less performant: every ETA change would have to wake up every monitor and have it do (largely redundant) date calculations. I obviously prefer the first approach if the timeline can be simplified enough.


    Challenge #1: Yes, the way to communicate between processes is via events. I don't know if I actually heard a specific question in your challenge, but events are the way to go. But this another reason why I don't like the monitoring process approach. Having every operational process report everything to the monitor process means that the monitoring process has a lot of hard calculations to do to figure out the meaning of those events. It's much easier to send business meaningful events into the operational process (like T-1H) and have the operational processes determine the right thing to do.

    Challenge #2: Yes, these' no perfect out of the box way to do this. (Which is why I do the whole dance above with IMEs.) The BPD system must be very efficient in determining the next thing to do. So there is a big event table in the DB that includes everything, including timer events that have been pre-calculated into actual date/times. That way the BPD system can just look at the database index and see the next thing to do (and whether it can be done yet). This oversimplifies a bit, especially around clustering, but that's the gist. If the BPM system had to re-calculate every timer based on the latest variable values every time it considered what to do next (or even every time a variable changed) the BPM system would be spending 99% of its CPU computing timers.

    Again, which leads me to the IME approach I took above. More or less you have to tell the system "something has changed such that you need to wake up and re-compute the timers". I like the first approach because it uses the timeline BPD to translate that into something business-meaningful, and incidentally reduces the numbers of timers that need to be re-computed. But either approach essentially gives the system a notification that times need to be re-evalauted.



    This is obviously just a seed of an approach. As I said, it's not a simple use case and I hope you have some experience under your belt. But it's already too long and probably too hard to understand. You could extend it to do things like change due dates for tasks, but I will leave that as an exercise to the reader.

    • AndrewPaier
      711 Posts

      Re: Task execution verification and timer variable changes

      ‏2013-07-12T20:52:34Z  in response to

      David's answer is excellent and very through.  I strongly agree with his feelings that separating the implementation from the monitoring is not a good approach and is likely to make your solution significantly more brittle.  

      One other option to throw out there for you in terms of your timer challenges is there is a little known (and lesser used) js call tw.system.rescheduleTimer(timerId, date).  This may be risky to use as I don't see any documentation of it officially from IBM, but it certainly was present and worked though the 7.5.1 release.  It also code completes on a JS block at the BPD level, but shows an error at the service level, so I didn't make the call up.  This call allows you to, as it says, reschedule a timer to execute at a different time than it was originally scheduled for.  This would take a little sophistication to get to work in your process but I wanted to mention it in case it was useful.

      I was hoping that what you could do was hand the timer ID in directly to the service as an input and then just do a reschedule, but that appears to throw an error saying that tw.system.step. "cannot read property from null" which surprised me because I thought the binding didn't happen until the task executed.  It turns out it seems that is not the case, as when I then assigned the timer ID to a variable on the pre of the timer I can see it is assigned to the variable at the BPD level, but when the service executes it has a blank value, so clearly the binding is when the task is created.

      So, given this situation, one option would be to also have an IME listener at the activity that could reschedule the timer.  That feels wrong model wise (putting technical details in the diagram) but would work.  

      I was hoping that the REST API might have the same calls, but while I found a fire timer call, I was unable to find a reschedule call.  At this point not even sure what I'm recommending.  The rescheduleTimer call is most frequently used when you want the timer to be repeatable and want to schedule it to fire again the the future.  The problem being that to make this work for your scenario (where the task has been started) you don't want to have the task closed, but in other scenarios you might.

      I guess one option would be to turn on tracking of the timer ID then you could query the performance server to get its value.  Would would then be able to reschedule the timer when the task is executed prior to the person seeing the UI.  That feels like the least bad solution using this API call.

      Andrew Paier  |  Director  |  BP3 Global, Inc.
      BP3 Global's Website  |  Twitter  |  Linkedin  |  Google+  |  Blogs

      • F7QD_Nicolas_Echavarria
        57 Posts

        Re: Task execution verification and timer variable changes

        ‏2013-07-22T01:13:30Z  in response to AndrewPaier

        David and Andrew,

        Thanks so much for your carefully taught responses!
        We are in the process of testing some of the approaches and will get back with our findings.

        Thanks again!!!

        Nicolas E.

      • RameshKumar C
        RameshKumar C
        25 Posts

        Re: Task execution verification and timer variable changes

        ‏2014-06-09T19:26:09Z  in response to AndrewPaier

        Hi Andrew,


        We got a request from Customer to reschedule the reminder timer in      
        Production for particular request.                                      
        We came across the below JAVA API which looks like to handle the above  
        request, but it did not work properly                                   
        tw.system.rescheduleTimer(timerId, newFireTime)                         
        timer id -                                      
        var date = new TWDate();

        I have run this api and I got null in inspector mode. So I came to know, it is working. But timer was not triggered at the specified time. Did I do any thing wrong here ? How to achieve this ? Regards, Ramesh

        • AndrewPaier
          711 Posts

          Re: Task execution verification and timer variable changes

          ‏2014-06-09T21:27:48Z  in response to RameshKumar C

          That is actually from the Javascript API, not java.  There is no supported Java API.  The code sample you uploaded makes no sense so I can't tell what you are doing or what may have happened.  I expect something more like -

          var myDate = new tw.object.Date();  //Creates a date set to right now.
          myDate.setHours(myDate.getHours()+4); //Sets the time to 4 hours from now.
          However what is important is how you are calling this, as I alluded to in my post it is easy to have null for the timer id which would make the code simply fail.  Generally this call has been used when you have a timer that you don't want to have interrupt the activity that you wish to reschedule to fire again in a set amount of time.  For example the code I wrote above could be used for a timer that you want to have fire every 4 hours.

          ANDREW PAIER
          T: 512.600.3239 X 240 / / @apaier
          BP3 /// / Blogs / Twitter / LinkedIn