Contestants on “Jeopardy!” have to understand the clues in light of the Category (context) and then quickly sift through their accumulated knowledge over their lifetime to come up with potential answers, decide if they have confidence in their answer, and then quickly respond by hitting the buzzer. Except for the buzzer, does that sound familiar and relevant to your job? Would a computer that could do all that plus cite the evidence backing up its answer be helpful to your job? If so, then read on.
At last week’s Analytics Solution Center June seminar, David Ferrucci from IBM Research described the IBM Research Grand Challenge of making a computer that can win at the game of “Jeopardy!”(For more on “Watson,” the computer behind the challenge, check out The New York Times Magazine cover article from last week)
Les Drieling, the keynote speaker and former US Government Intelligence Agency senior scientist (now an IBM executive in the Global Business Services NISC business unit), highlighted the needs in the intelligence community to sift through very large volumes of structured and unstructured data with the goal of making very important (some life and death) decisions under extreme time pressure. It might seem obvious that the DeepQA technology behind the “Jeopardy!” project can be used to help intelligence analysts to filter and retrieve information using natural language queries.
But don’t many agencies have the need to retrieve
information quickly and precisely in response to questions? For example, the Coast Guard may have wanted
to know what dispersants are best for using in the open ocean to fight the
The open government movement is also spurring use of this sort of technology to help its citizens. For example, a citizen could use it ask about the status of a new law or who to contact with regard to a particular problem.
Let’s say the citizen’s question was “How to get my car emissions tested when I’m away at college out of state and this state doesn’t do emissions testing?” A question & answer system would have to be able to decompose the query, search its database for possible answers, and then select the best answer to return to the citizen. If all the questions are known ahead of time, then government personnel can develop a simple look-up system. However, when there are so many questions that they can not all be itemized ahead of time, then a more sophisticated approach is required. This is where the IBM DeepQA technology comes in to play.
Text Analytics underlies DeepQA as well as the other presentations
at the June Analytics seminar. Another presentation showed how the National
Highway Safety Administration (NHSTA) Defects and Recalls database could use a
text analytics tool to alert on the possible connection between Toyotas and
rapid acceleration much earlier than this pairing came to the NHTSA, or the
public’s, attention. The tool used for
this demonstration was the IBM
Content Analyzer and it allows enables you to search, discover, and perform
the same analytics on your textual data that is done with structured data. In the demonstration it identified the
unusual relationship between “
Another example showed how an Intelligence Agency was using text analysis to analyze the performance of its counter terrorism efforts. In this example, report text was analyzed and scored to determine whether their objectives were being met and which intelligence methods were most helpful in meeting their objectives.
Finally, IBM showed how text analytics could be used to discover major themes that occurred when USAID (US Agency for International Development) ran a “Jam” that asked participants from around the globe to propose “pragmatic ideas and solutions to some very real issues and problems facing our communities and our world today” (this is from the Global Pulse 2010 Website). The “Jam” tools quickly identified and classified the participant’s key ideas in real time so that later participants could join the conversation on their topics of interest.
The Category is “Government.” For $100, the clue is “Text Analytics.” <Buzz> “What can help the Government make better and faster decisions from our mounds of unstructured data?”
To see the charts and listen to the replay from the seminar go to www.ibm.com/ASCdc and look under past events.
Give me your comments on how this technology can help you and
your agency or write to me at
- Frank Stein, Director of IBM's Analytics Solution Center, Washington, D.C.