Cloud Computing

Big Data, Epistemology, and IBM Watson

Share this post:

Scottish Law has always been a puzzle to me.  There are some odd and arcane aspects to it that seem more in keeping with Masonic ritual than public justice, yet they persist to this day.  Take the Society of Writers to Her Majesty’s Signet, an organization of Scottish solicitors that had some special privileges since the late sixteenth century.  One such Writer to the Signet was a man called John Ferrier, who married a sister of the writer and philosopher John Wilson. They had a son, James Frederick, in 1808, who followed a new and emerging career path, and became a meta-physicist.

So what has all this got to do with, well, anything? Well, James Frederick Ferrier was the man who is credited with the phrase epistemology, from the Greek episteme (knowledge) and logos (study).  It refers to the study, or theory, of knowledge; or what the Stanford Encyclopedia of Philosophy refers to as justified belief.  Why, it could be put, do we believe what we do?  The thought occurred to me as I considered the best way to answer a question that was put to me by a client in South America – so what’s the difference between Siri and Watson?

Siri Versus Watson

Sometimes Siri gets confused

The immediate answer is to do with scale, breadth, accuracy.  And this is certainly true.  Watson has an extraordinary capability to answer a vast array of questions, trawl through millions of documents for possible answers, score possibilities, and make a smart determination about the right one, really fast.  Siri, on the other hand, often has hard coded answers to certain questions (such as “What is your favourite phone?”), is linked to a very small set of APIs (like Maps), and has a very shallow appreciation for context.

A more serious question might be what the difference is between Watson and Google.  Let’s assume that there are no differences in terms of scale and breadth.  Google clearly has an astonishing capability to find things on the Internet.  But what about accuracy?  As a commercial search engine, Google certainly needs to be accurate, otherwise people are simply not going to use the search engine.  If people don’t see relevant results, they are not going to click, and if they don’t click, Google doesn’t get paid.  But there is an accuracy threshold that Google needs to attain in order to remain viable, that we can call a commercial standard.  Google doesn’t always find what I’m looking for, but it’s good enough that I’ll return there the next time I’m looking for something.

Watson, on the other hand, will refuse to answer a question if it thinks it doesn’t know.  Watson doesn’t guess.  Watson tries really hard to understand context, and within that context, apply various hypotheses and justifications to its approach to answering the question.  While the Google commercial standard may work for Google, the Watson standard becomes relevant for much higher objectives – like medical diagnoses.

It would have been fascinating to get Ferrier’s take on Big Data.  There is so much accessible data in the world today, much of it conflicting, there are real challenges emerging in determining not just data, or knowledge, but a kind of justified belief.  We know so much, in many cases, that we know almost nothing.  We are of course talking less about search, and more about artificial intelligence, that helps us to understand that – as I mentioned to my friend in South America – if I am looking for the girl from Ipanema, I may actually be looking for an MP3 download, and not for a lady who happens to live just south of the Copacabana.

So, next time I’m asked the question about the difference between Siri and Watson, my answer will be simple.  Watson concerns itself with epistemology; Siri does not.


Digital Transformation Segment Leader, IBM Global Telecommunications Industry

More stories

If automation is the end game, then machine-enabled automation is a winning hand

Moving to cloud-based networking with network function virtualization (NFV) and software-defined networking (SDN) demands new ways to enable operations and service lifecycles across hybrid networks. The ability to operate and orchestrate the network through machine learning and artificial intelligence, combined with automation is a key requirement to delivering on the promised business benefits of network […]

Continue reading

IoT: Bridges and the bronze medal

In 2018, the word “connected” is used in every industry – be it personal, manufacturing, mobile or global. The image of connected conjures thoughts of technology making people and things more accessible. In the case of IoT, the purpose is to provide insightful information that enables individuals and organizations to make higher quality decisions. What […]

Continue reading

The race to scale network virtualization through automation is accelerating

Telcos, buckle up: The race to bring virtualized networks to scale is speeding up. The idea of network function virtualization first emerged in 2012 as a way for telcos to meet rising demand for bandwidth, cut down costs, and speed up delivery of new services. But for the most part, implementations have been slow and […]

Continue reading