Baruch Schieber and I just published a point of view on analytics that can be found here.  This paper isn't a technical paper.  It is aimed at executives who want to know the essentials about a given topic, analytics in this case.

Baruch works at IBM Research where he is managing the optimization center.  I work at IBM software group where I lead the use of analytics within industry solutions.  Given analytics is covered in many papers, blogs, books, etc, we tried to shed some lights on topics that aren't t discussed that much.  We focussed on what executives new to analytics should worry about.  We assumed analytics value was understood, and we listed a number of issues that must be taken care of in order to harvest the value analytics could provide. 

Here is the list we ended up with:

• How can you efficiently use uncertain or incomplete data in an analytics application?
• How can you design an effective decision-making system when one part of that system includes human interaction?
• How can you make analytics socially and organizationally acceptable?
• How can you grow the number of people with the correct skills to apply analytics to a business problem?
• What learning tools, sandbox environments, education, and academic partnerships to train students or other resources need to be devised?
• How can you simplify the modeling process by using more powerful tools?

I hope these are self explaining, but I'd like to give a concrete example of the third one.  I once was working for a railway company as an ILOG consultant.  The customer wanted to automate the computation of locomotive drivers rosters.  It meant that the purpose of the application we were developing was to define who drives what, and where do they spend night. 

The first system we delivered was computing drivers schedules while meting all labor rules and other constraints in a reasonable amount of time.  Yet it wasn't really used in practice.  When we asked why, the answer was enlightening: "well, some drivers want to have a night stop at a given location, in order to meet someone else than their spouse...  Others prefer to be back home every night."  Stating such rules explicitly in the system was violating some privacy law, hence wasn't part of the mathematical model of the problem. 

 

The cure was to allow an interaction between the human decision maker and the optimization system.  We implemented a way to edit the solution by human planner.  It was then possible for the planner to add manually some explicit overnight stops for some drivers, and have the problem solved again.  It was also possible to force some night stay home for others, and have the problem solved again.  This way the planner was able to cope with constraints that couldn't be stated explicitly in the mathematical model.  Whenever the proposed solution wasn't meting those implicit constraint, the planner was adding some required stops to the model, then asked for a new solution.  The process was iterated until the planner was pleased with the solution.

 

This new way of computing rosters was well received, and it had bee used for years.  We see two reasons for the better acceptance.  First,  important, but implicit, constraints could be taken into account.  Second, there was someone to go to if the proposed rosters didn't suit some driver.  Unhappy drivers could go talk to the planner and ask for  change, and could sometimes get a change done.  This wasn't possible with a fully automated system.  With an automated systems, unhappy drivers would not know who to talk to in case of disagreement.

This example shows that  a change in the way analytics is applied can have a dramatic effect on its acceptance.  The example was in the field of mathematical optimization, but similar situations happen whenever analytics is used tot make automated decisions.  The ability for a human to overwrite a proposed decision is sometimes key to acceptance.

 

The other challenges are worth discussing and I may blog about these later.  Stay tuned!  And in the meantime you can download that point of view here.