IBM computer system a scientific achievement in natural language processing

Share this post:

Editor’s note: This is a guest post from Charles Lickel, Vice President, Software, IBM Research.

The IBM Watson computer system, named after company founder Thomas J. Watson, Sr. catapults data analysis by understanding the nuances of “natural language” that humans use every day.

A breakthrough in the scientific field of DeepQA – a field aimed at furthering computing intelligence – when we began this project more than four years ago, others in the scientific community believed this task to be impossible and we were uncertain about what we would be able to achieve. In fact, it took our research team two years to get the system to be able to analyze a Jeopardy! clue and provide an accurate response in less than three seconds. That was the moment we knew we had a chance of achieving a breakthrough in artificial intelligence and natural language processing, as well as a chance to compete against Jeopardy! champions.

Watson uses Unstructured Information Management Architecture (UIMA), IBM’s open-source framework for analysis of unstructured content, to understand natural language text, speech, images and video. The complex algorithms for the analytics engine behind Watson are tuned specifically for “open-domain” QA – covering a wide domain of knowledge including history, geography, arts, science, sports, and popular culture. It helps Watson handle the broad range of information that language can express and evaluate the “evidence” it collects to determine a confidence level to answer a question.

This fall, the Watson system and the Research team achieved remarkable results, when Watson played more than 55 “sparring games” against former Jeopardy Tournament of Champions contestants.

Watson’s ability to understand the meaning and context of human language, and rapidly process information to find precise answers to complex questions, holds enormous potential to transform how computers help people accomplish tasks in business and their personal lives. Watson will enable people to rapidly find specific answers to complex questions. The technology could be applied in areas such as healthcare, for accurately diagnosing patients, to improve online self-service help desks, to provide tourists and citizens with specific information regarding cities, prompt customer support via phone, and much more.

Like Deep Blue, the IBM supercomputer that defeated the reigning world chess champion in 1997, Watson represents a major leap in the capacity of information technology systems to identify patterns, gain critical insight and enhance decision-making despite daunting complexity. But while Deep Blue was an amazing achievement in the application of compute power to a computationally well-defined and well-bounded game, Watson faces a challenge that is open-ended and defies the well-bounded mathematical formulation of a game like chess. Watson has to operate in the near limitless, ambiguous and highly contextual domain of human language and knowledge.

Win or lose, Watson will open eyes to what kinds of questions a computer can answer, and open doors to what kind of problems a computer can solve. For more about the technology behind Watson, visit ibmwatson.com.

More stories

A new supercomputing-powered weather model may ready us for Exascale

In the U.S. alone, extreme weather caused some 297 deaths and $53.5 billion in economic damage in 2016. Globally, natural disasters caused $175 billion in damage. It’s essential for governments, business and people to receive advance warning of wild weather in order to minimize its impact, yet today the information we get is limited. Current […]

Continue reading

DREAM Challenge results: Can machine learning help improve accuracy in breast cancer screening?

        Breast Cancer is the most common cancer in women. It is estimated that one out of eight women will be diagnosed with breast cancer in their lifetime. The good news is that 99 percent of women whose breast cancer was detected early (stage 1 or 0) survive beyond five years after […]

Continue reading

Computational Neuroscience

New Issue of the IBM Journal of Research and Development   Understanding the brain’s dynamics is of central importance to neuroscience. Our ability to observe, model, and infer from neuroscientific data the principles and mechanisms of brain dynamics determines our ability to understand the brain’s unusual cognitive and behavioral capabilities. Our guest editors, James Kozloski, […]

Continue reading