Contestants on “Jeopardy!” have to understand the clues in
light of the Category (context) and then quickly sift through their accumulated
knowledge over their lifetime to come up with potential answers, decide if they
have confidence in their answer, and then quickly respond by hitting the
buzzer. Except for the buzzer, does
that sound familiar and relevant to your job? Would a computer that could do all that plus
cite the evidence backing up its answer be helpful to your job? If so, then read on.
At last week’s Analytics
Solution Center June seminar, David Ferrucci from IBM Research described
the IBM Research Grand Challenge of making a computer that can win at the game
of “Jeopardy!”(For more on “Watson,” the computer behind the challenge, check
New York Times Magazine cover article from last week)
Les Drieling, the keynote speaker and former US Government Intelligence
Agency senior scientist (now an IBM executive in the Global Business Services NISC
business unit), highlighted the needs in the intelligence community to sift
through very large volumes of structured and unstructured data with the goal of
making very important (some life and death) decisions under extreme time
pressure. It might seem obvious that the
DeepQA technology behind the “Jeopardy!”
project can be used to help intelligence analysts to filter and retrieve information using natural
But don’t many agencies have the need to retrieve
information quickly and precisely in response to questions? For example, the Coast Guard may have wanted
to know what dispersants are best for using in the open ocean to fight the Gulf Coast
oil spill, what evidence exists as to their efficacy and safety, and a
confidence value on the proposed answer.
The open government movement is also spurring use of this
sort of technology to help its citizens.
For example, a citizen could use it ask about the status of a new law or
who to contact with regard to a particular problem.
Let’s say the citizen’s question was “How to get my car
emissions tested when I’m away at college out of state and this state doesn’t
do emissions testing?” A question &
answer system would have to be able to decompose the query, search its database
for possible answers, and then select the best answer to return to the citizen.
If all the questions are known ahead of time, then government personnel can
develop a simple look-up system.
However, when there are so many questions that they can not all be
itemized ahead of time, then a more sophisticated approach is required. This is where the IBM DeepQA technology comes
in to play.
Text Analytics underlies DeepQA as well as the other presentations
at the June Analytics seminar. Another presentation showed how the National
Highway Safety Administration (NHSTA) Defects and Recalls database could use a
text analytics tool to alert on the possible connection between Toyotas and
rapid acceleration much earlier than this pairing came to the NHTSA, or the
public’s, attention. The tool used for
this demonstration was the IBM
Content Analyzer and it allows enables you to search, discover, and perform
the same analytics on your textual data that is done with structured data. In the demonstration it identified the
unusual relationship between “Toyota”
and “acceleration” – automatically.
Another example showed how an Intelligence Agency was using
text analysis to analyze the performance of its counter terrorism efforts. In this example, report text was analyzed and
scored to determine whether their objectives were being met and which intelligence
methods were most helpful in meeting their objectives.
Finally, IBM showed how text analytics could be used to
discover major themes that occurred when USAID (US Agency for International
Development) ran a “Jam” that asked participants from around the globe to
propose “pragmatic ideas and solutions to some very real issues and problems
facing our communities and our world today” (this is from the Global Pulse 2010
Website). The “Jam” tools quickly identified and classified the participant’s
key ideas in real time so that later participants could join the conversation
on their topics of interest.
The Category is “Government.” For $100, the clue is “Text Analytics.” <Buzz> “What can help the Government make better and
faster decisions from our mounds of unstructured data?”
To see the charts and listen to the replay from the seminar
go to www.ibm.com/ASCdc and look under
Give me your comments on how this technology can help you and
your agency or write to me at email@example.com.
- Frank Stein, Director of IBM's Analytics Solution Center, Washington, D.C.