What’s next for Watson?
daryl_pereira 270002AW8D Visits (2804)
(Guest post by Sarah Duffy, WebSphere Web Strategist)
In an almost packed room with 35+ attendees, Marshall Schor, who was a key team member of the systems team for Watson participated in an informal Q&A session during the WebSphere Unconference at #IBMImpact.
During this 45 minute session, Marshall discussed the development process, challenges and what’s next for Watson.
I thought it was interesting that it involved three to four years to develop Watson but also took a significant amount of time to negotiate with the Jeopardy producers to accept Watson as a valid contestant.
Written in JAVA, C and C++ with 64bit Java, Watson was the largest test case the development team could have ever imagined. If you think about a “typical” jeopardy contestant, they are born, learn, read, and become a contestant on the show. In Watson’s case the team was responsible collecting half of a terabyte of data. The majority of the data is articles, text, and image arrays including the NY Times archives beginning in the early 1900s and the Bible for example. “We didn’t say, suck in the Internet” said Marshall.
Watson needed to learn how to answer different types of questions, for example:
If Watson pulled up an incorrect response, the team would make adjustments in the tables. They would run another test looking for improvements in the results. Part of the process every week was to complete a test run and receive a few gigabytes in results to complete error analysis to see what occurred in previous runs and current runs. And then continuing the process to see if it improves the weekly score.
A few challenges the team faced included:
So what’s next for Watson? His calendar is booked on tour throughout the end of this year. The team is also looking into what Watson can to accomplish within businesses, for example 1- medical diagnosis – teamed with physicians at Columbia and John Hopkins to repurpose for medical area. 2- General read this and answer these questions. Statement or facts that are only true in certain situations.