# A short history of analytics

In the late 1700s, William Playfair invented the bar and pie charts and in doing so created a new discipline that we all benefit from today: statistical graphics.  When we talk of “analytics” and “business intelligence” we are all following in his footsteps.  Every time we look at a brand scorecard we have him to thank (or not).

As a side note you may be interested to hear that Playfair was both a scoundrel and a genius.  His CV includes engineer and inventor, silversmith, investment broker, economist, publicist, land speculator, convict, blackmailer and journalist.  He even stormed the Bastille in Paris!

Playfair’s breakthrough was a result of a lack of data (think of how different the context is today).  He wanted to model trade data as time series, but couldn’t find the data on Scotland so he made the mental leap to a visualization that wasn’t tied to time or space – it was pure comparison of structure, quantitative data, and that is the rut that analytics business intelligence has been stuck in ever since.

# The past and the present of analytics are colliding

So much of what is meaningful in the world is not measured by numbers or easily plotted on charts or analyzed with classical statistics.  The data is unstructured, messy and very, very big – text, images and numbers all vie for attention.  It takes an act of cognition, of creative interpretation (“intuition”) to make the data talk.

Of course, human beings have been guessing for millennia, but this is not what I am talking about.  There is a creative partnership between hard, factual data, the more qualitative forms of analysis that go with unstructured data, and the human mind.

Imagine the scenario that your life depended upon correctly estimating the weight of an ox that is standing in front of you. You have no weighing scale. Can you do better than a random guess?

In 1906, the great statistician Francis Galton (with whom I share a birthplace) observed a competition to guess the weight of an ox. 787 villagers entered the competition, but no individual contestant correctly guessed the weight of the ox at 1,198 pounds. However, Galton discovered that the average guess (1,197lb) was very close.  James Surowicki picked this story up in his famous book The Wisdom of the Crowds but here I want to use it to illustrate a different message.

These were country folk and they understood the context for the competition and, in their local dialect, they probably talked about ways to get to the right answer.  No doubt the people sizing-up the ox were searching their experiences and memories for useful analogies, anecdotes, images of oxen from previous competitions, salient facts and figures (“evidence”).  Then maybe they generated and evaluated an evidence-based hypothesis about the weight of the ox and discussed that with others (but only those they could trust – it is a competition after all).  Finally, they probably refined their estimates based on conversations and intuition.  Not guessing as many would think of it.  And, as Galton found out, if they had talked collectively and pooled their insights they would have got the right answer.

# Why am I telling you this?

The digital transformation of FMCG companies is underway, and there is a rising passion for data-driven decision-making in the industry.  FMCG companies in parallel are entering (late) into a new era of computing that IBM call cognitive-computing (think of it as a very weak form of Artificial Intelligence or AI).  It has the potential to change the way we think about and use analytics (for example, look at how Campbell Soup is using weather data, analytics and cognitive computing to activate their brand).  We need these new tools to personalize, make programmatic work better, and to place ads where people will actually see them.

Traditionally computers have required programming, logic and rules to make them work. They can handle numbers but human readable information (images and text) is much more difficult. With a technology called Watson, IBM has overcome many of those limitations and created a system that understands natural language, can interact more naturally with people and adapts and learns from experience.  Isn’t that just what the competitors seeking the weight of the ox were doing?

Analytics are a powerful way to make sound decisions, make predictions and identify the next best action.  However, in the past, analytics tools have been hard work, requiring specialists to extract insights (often too slowly to be of use).  Typically they are unable to work with the most meaningful data (that messy unstructured stuff I mentioned at the start).  This might be why only a fraction of businesses use powerful analytics tools as part of their decision making.

Today over 85% of data is unstructured. This data includes your consumers talking about your product, their needs, hopes and how and why your product might not have lived up to their expectations. Analytics powered by cognitive computing enables businesses to use all of the big data in their ecosystem as the basis of competition advantage.  Remember, half of the world’s data was created in just the last ten months. Let that sink in… half of the world’s data in the history of mankind was created in less than a year. And the growth is only getting started with data growing to 44 zettabytes by 2020, a 53-fold growth from the beginning of 2010.  Yet in 2014 only 0.5% of all data was analyzed and used.

Yes, Playfair and his statistical graphics (“descriptive analytics”, “predictive analytics” etc) will continue to contribute much to our ability to understand our world, but a new generation of tools that bring the old world of statistical modelling together with cognitive computing in the cloud (e.g. Watson Analytics) will help people make better decisions by augmenting the best qualities of human beings in order to create insights at speed and scale.

# The near future for analytics

Business have just started to realize the full potential of analytics for their businesses. Over the past year, the number of organizations with data-driven projects has increased by 125%. But just as businesses are getting their heads wrapped around cognitive analytics, the next wave of analytics tools is being released onto the market as the digital world does not stand still.

Take for example Project DataWorks, a new cloud-based platform from IBM.  It is the first to integrate all types of data and use cognitive computing for analytics alongside classical statistics. It can ingest data faster than any other data platform, from 50 to hundreds of Gb/s, swallowing whole lakes of data from sources including enterprise databases, the internet of Things (IoT) and social media. These technologies are amongst the first to open up the new world of edge analytics, for example, that will be so important in consumer homes and cars.

New technologies such as DataWorks aim to foster collaboration and give users self-service access to sophisticated data and models (tapping their inner data scientist perhaps).  The exotic technologies involved (Spark, Cassandra etc) are obscured from the user- its all about making the experience seamless and playful.  For the more adventurous have a look at IBM Data Science Experience as a place to play with analytical code and to visualize big data.

# Challenges for FMCG (and everyone else)

Behavioral economics and psychology research of the past 50 years has shown that humans are far from rational decision makers. We are heavily influenced by context and our (positive or negative) experiences. Even getting the right data insights to assist our decision making does not eliminate our decision making flaws (e.g. Anchoring). One could argue that the 2008 global financial crisis was based on our inability to make rational decisions in certain situations. Thus, maybe we need to rethink which decisions would be better in the ‘hands’ of a machine…?

What do you think?  As we get closer and closer to an always-on, real-time world of streaming big data, are machines better placed than us to answer questions, such as, “do consumers like my new product?” or “what is the next big food trend?”. Or is it Manus x Machina as we demonstrated at the Met Gala earlier this year?

I say it is a partnership we are looking for between our smart tools and ourselves. Collaboration in the broadest sense.  And to truly exploit the potential of these new analytics tools, the right (“golden”) questions need to be asked, and that can be challenging for those of us brought up in Playfair’s more straightforward world.  We need to find our inner ox-weight-guesser!  Babs Rangiah, Partner, Global Marketing Solutions iX at IBM, says “the skills gap may be the biggest hurdle holding companies and this industry back from capitalizing on what should be a revolutionary period in marketing”.  The same applies to manufacturing, supply chain, merchandizing etc.

I would also argue that as data is moving to the heart of business strategy, different types of people are required to work with data on a daily basis. Only then will companies  become effective in working with these tools.

Perhaps the greatest challenge for businesses that place the consumer and the resources of the planet at their centre will be ethical.  Just because we can analyze and take action (for example automatically send a consumer a marketing message because we think they have had a child), doesn’t mean we should.  That’s why industry leaders have established a partnership on AI Best Practices, to ensure that future systems express their actions based on values that we would recognize as aligned with human intentions.

# A final thought about analytics

Cognitive computing has already birthed a new and more powerful wave of analytics and taken us a big step forward on the road to AI.  Take a photo of someone and a cognitive system can probably work out who it is, then aggregate their social profiles to create a psycho-demographic profile, match it with a high degree of certainty to their retailer loyalty data, send them a tailored promotion and engage them in a simple Q&A via text. Today.  I cannot wait for tomorrow.

IBM, Distinguished Engineer

Previous Post

Next Post

More stories

Wonder-ing out of the uncanny valley The “uncanny valley” is a phrase used to describe machines or systems that at the same time appear close to human, but elect a sense of revulsion rather than wonder.  Creepy is another word that gets used.  It is an old idea (first used by a Japanese roboticist working […]