For “That’s Debatable,” Watson analyzes every submitted argument to determine if the content is for or against the position statement. Watson also removes submissions deemed irrelevant, such as those that are off-topic, or those that are deemed neutral – neither supporting nor contesting the statement.

Watson evaluates the quality of each submitted argument and identifies potential key points by grading and filtering high-quality arguments. It disregards potential key points that are too long, too emotional in tone, are incoherent or include redundancies. The arguments matched to each key point are sorted by the their argument quality and by the confidence in the matching.

The ranking of the quality is based on the machine learning model, which was trained on human assessments of over 30,000 arguments. There is a general correlation between the argument quality score and factors including grammar, clarity and repetition.

Watson now identifies how many arguments support each of the potential key points. It then selects a small set of key points that are diverse and cover the majority of arguments submitted.

Finally, Watson selects the key points cited most often in the submissions. For each key point, a small subset of the strongest arguments that support that key point is selected. Those key points and arguments are then used to formulate coherent, salient narratives — one arguing the pro side of the debate, the other arguing the con side.