Natural language poses unique challenges for AI. Although language is governed by rules meant to dictate grammar use and spelling, these rules are not always followed, and different languages have different rules. Even when the rules are followed, the result can often have ambiguous meaning. For a natural language processing (NLP) system to master natural language, it must be able to both generalize and reason over new text and recognize relationships between different words in context.
IBM is a leader in NLP technologies that enable computer systems to learn, analyze and understand sentiment, dialects, intonations and other aspects of human language. Our research teams are responsible for developing many of the NLP capabilities IBM has brought to market, including those in Watson Discovery for document understanding, Watson Assistant for virtual agents and Watson Natural Language Understanding for advanced sentiment analysis. IBM Research is also behind the technology IBM plans to release to help businesses recognize and extract information in complex business documents such as contracts.
IBM Research’s contributions to NLP extend far beyond our own offerings. We play a leading role in making key language understanding benchmarks available to the larger research community in areas such as question answering, task oriented-dialog and summarization. We actively contribute data sets to researchers outside our organization, and our data sets represent the realistic challenges that enterprises face, as opposed to more generic benchmarks typically found in open domain data.
As one example, our TechQA data set includes 600 training, 310 dev and 490 evaluation question/answer pairs—actual questions posed by users and answered by experts in an IT support domain. The questions and answers are likewise longer, with a median length of 35 words as opposed to a median length of 12 words as typically found in current leaderboards. We created this dataset by crawling the IBM Developer and IBM DeveloperWorks forums for questions with accepted answers that appear in a published IBM Technote—a technical document that addresses a specific technical issue.
IBM Research’s Project Debater plays a crucial role in advancing state-of-the-art NLP. In June 2018, we introduced Project Debater to the world in the first ever live, public debates between AI and humans. For many people, that performance was an eye-opener, as we demonstrated AI could quickly build a factual argument, consider a counter argument and deliver a rebuttal—all in the same natural language we use every day. We saw Project Debater’s unveiling as just the next step—albeit an important one—on a journey to deliver AI with robust Natural Language Processing (NLP) capabilities to businesses everywhere.
With today’s announcement that IBM will begin integrating NLP features developed for Project Debater into Watson, IBM Research once again delivers unique technology from the lab to the enterprise.
In the coming months, IBM clients using Watson Discovery, Watson Assistant and Watson Core Services will for the first time be able to take advantage of advanced sentiment analysis, new summarization capabilities, advanced topic clustering and customizable classification of elements in business documents. The new advanced sentiment analysis feature, for example, will enable Watson APIs to now identify and analyze idioms (“up in the air,” for example) as well as colloquialisms used in informal speech.
Our goal is to offer IBM Watson clients the ability to apply Project Debater’s NLP capabilities to everyday tasks, whether it is assisting attorneys with trial preparation, promoting more civil discourse in online forums or identifying financial facts that either support or undermine an investment strategy.
Project Debater at Cambridge
As we’ve developed Project Debater’s capabilities, we documented the technology’s maturation over time through public demonstrations. At the January 2019 CES conference, we demonstrated Project Debater Speech-by-Crowd decision-support capabilities. The following month, at our 2019 THINK conference in San Francisco, IBM Research hosted a second live debate for Project Debater, to test its growing skillset against champion debater Harish Natarajan. In November at the world’s oldest debating society, Cambridge Union, Project Debater augmented two debate teams as they squared off by providing them arguments submitted by the public. This underscored, once again, how the technology can work alongside people — and assist them.
Looking ahead, our researchers will continue to develop Project Debater’s core NLP capabilities, improving the technology’s ability to, for example, pinpoint relevant material to establish a good argument and better reason in favor of or against a particular issue. IBM Research AI’s NLP strategy will continue to support Watson’s, with our breakthroughs focused on all elements of mastering natural language — understanding, classification, retrieval, and generation.
Watch The Debater
Check out the trailer for “The Debater,” a rare, behind-the-scenes look at the making of Project Debater through the lens of an eclectic team of researchers that dare to take AI into uncharted territory. Official Selection of the Copenhagen International Documentary Film Festival.