JeanFrancoisPuget 2700028FGP Visits (9018)
In my last post I discussed how gamification could be used to overcome the resistance to automated decision making systems. The case discussed in my previous post was about a system that computes retail prices for hotel rooms. The point of gaming was to show that human intervention would degrade the business outcome. Prices set by humans lead to less revenue than prices set by the system.
Interesting comments on that post made me realize that I have been a bit extreme in my will to make a point. While I stand by my previous blog content, truth is that gaming will not overcome all resistance to adoption of automated decision systems. We have seen cases where users of automated systems were focusing their efforts in finding flaws in the system rather than trying to leverage it. As a result, such systems were abandoned; While this extreme resistance may not the most common case, it happens.
Based on our rather extensive experience in delivering decision making systems, we have developed an approach that improves buy in from users. The key point is to move away from a pure batch process to an interactive decision making process, where human and machine collaborate to reach the best decisions. With this approach, human intervention improves the business outcome compared to a fully automated system.
How come I now say that human decision makers may improve upon solutions generated by mathematical optimization when I said the exact opposite above? Key is to realize that very often the problem solved by optimization is different from the actual business problem.
Indeed, he application of optimization to a business problem requires a translation of the business problem into a mathematical optimization problem. This translation process is called modeling.
Modeling often requires an experienced analytics practitioner, for instance a PhD in Operations Research. Modeling requires a skilled person because the translation may not be obvious. it is not obvious because it is a tradeoff between :
There may be other reasons for not including some of the business problem into a mathematical problem, legal ones in particular. I was for instance involved in the development of a train attendant scheduling application. We could not include, by law, personal data (constraints) such as the fact that employees X and Y do not go well together hence should not be scheduled on same trains. Another example was that Mr Z liked to stay overnight away from home for reasons I won't clarify here...
As a result, the optimization problem often ignores some of the business problem. Human intervention is then required to adapt the output of optimization in order to take into account what was left out of the optimization model. In my train attendant scheduling application, a manager have to manually edit whatever solution optimization provides in order to take into account personal data.
We can therefore handle all the peculiarities of the business problem with a three step process:
Issue is that the end result may be quite suboptimal as explained above.
A better way is to change the way we approach the problem solving process. We can look at it as a cooperative process where optimization is used to solve a subset of the problem, namely the part that is modeled as an optimization problem, and the human handles the full business problem. The decision making process is then an iterative one where the mathematical problem is modified until its solution looks good to the human decision maker.
The process then becomes:
For instance, in the train example, a manager would be able to add additional constraints if some of the non modeled constraints are violated. If employees X and Y happen to be on the same train in the mathematical solution, then the manager would modify the problem by moving one of X or Y to another train, and adding this assignment as a constraint to the optimization problem. Similarly, if Mr Z does not stay overnight, the manager could add the number of overnights for Mr X as an objective to be maximized. Modifying the optimization problem that way is not unlawful as long as the personal data that drive these modifications isn't stored in the system.
The core of the process is therefore to solve variants of the optimization problem. These variants are often called scenarios. They correspond to slightly differing sets of hypothesis and constraints about the business problem to solve. The iterative process is therefore to explore various scenarios and select the one that gives the most appealing solution. Given user try various scenarios this process is also called what-if analysis.
In order to make this process effective, the decision maker must be able to easily create new scenarios from existing ones and compare their outcomes. This is complex as creating a scenario means a change in the way a business problem is modeled as an optimization problem. We explained above that this typically requires a PhD in operations research. It means that the decision maker would need such PhD to explore various scenarios. This is doable, but not very agile nor scalable.
A different way of doing this is supported by our Deci
The net effect of this interactive process is that the human decision maker has the final word. This is very different from using fully automated systems where no room is left for human decision making. The former is much easier to adopt. The latter should be used only when what is being optimized is something that does not require human intervention, like scheduling a semi conductor manufacturing plant, or trade settlement.