How Watson is helping clinicians and cancer centers to identify more patients for clinical trials

A Q&A with Andrew Scott, Watson for Clinical Trial Matching offering manager

Q1) What does Watson for Clinical Trial Matching do and who uses it?

IBM Watson® for Clinical Trial Matching uses artificial intelligence (AI) and natural language processing technology to help cancer centers identify potential candidates for trials and help clinicians find potential clinical trials for their patients. It integrates directly into the electronic medical record (EMR) so it fits within the regular workflow of oncologists, clinical trial coordinators and research nurses in academic medical centers, cancer centers and integrated delivery networks.

Q2) How does Watson for Clinical Trial Matching fit into the typical workflow of clinicians and clinical trial coordinators?

Watson does the work that would typically take hours to complete and uncovers data that might be missed if you were managing trial recruitment manually. It is designed to make the process more efficient and gives users the ability to screen more patients.

As a clinician, if I’m in a patient’s EMR , I can easily launch Watson for Clinical Trial Matching and quickly get a list of the trials that may be suitable for my patient. Equally as important, I also get a list of the trials that my patient has been excluded from based on the eligibility criteria. On the flip side, clinical trial coordinators can use Watson for Clinical Trial Matching to produce a list of potential candidates for their trials to help them meet enrollment targets. This is managed through a task board called “Tracker” that surfaces the list of patients and provides the trial coordinator with a convenient way to manage their trial recruitment.

Q3) How is AI used in Watson for Clinical Trial Matching and why is that important?

In most areas of medicine, a lot of patient information is hidden in unstructured data in the EMR, so finding relevant data fast is extremely difficult. But Watson’s natural language processing allows it to read and understand patient notes and other relevant documents to uncover those very specific pieces of clinical data needed to match a patient to eligibility criteria. Similarly, Watson uses natural language processing to read and understand trial eligibility criteria and quickly provide a detailed evaluation of a patient’s potential eligibility for the trial.

Some of our competitors say they are applying AI, but they’re only focused on natural language processing of patient data. We’re pretty unique with the breadth and depth of our AI capabilities, in that we’re not only teaching Watson to understand patient data, but also to understand clinical trial eligibility criteria. Competitors simply run high-level queries on and can end up with a list of 50 to 75 trials for further review. The research nurse or clinical trial coordinator then has to comb through that list to see which trials might truly be a good fit. That’s still a massive task. In contrast, it has been shown that Watson’s artificial intelligence does an in-depth review of the eligibility criteria to produce a much more precise list upfront.

Q4) What data sources does Watson for Clinical Trial Matching pull from and what information does it provide?

Watson ingests information from, which is an internationally recognized database that includes more than 95 percent of trials open for recruitment. This is automatically updated every week. The site then has access to all of the key clinical data of the trials, including the detailed eligibility criteria. Our patient data comes from direct integration with a data source at the site, typically the EMR. This is where Watson applies its natural language processing to find the key clinical attributes of the patient and create a comprehensive clinical profile. Every night, Watson refreshes the patient data, which means it will update any changes to existing patients and flag any new patients that become eligible. This means that clinical trial coordinators are made aware of any patients that are newly eligible for their trial. From this, Watson produces either a ranked list of potentially relevant trials an individual patient or a list of potentially eligible patients for a clinical trial. But that’s not all it provides. In both cases, Watson provides an assessment of each of the eligibility criteria of the trial to help the clinicians in their review and decision making.

Q5) How does Watson for Clinical Trial Matching differ from a clinical trial management system (CTMS) or electronic data capture (EDC) system?

They’re actually complementary with each other. Watson for Clinical Trial Matching is focused on screening and identifying patients for trials and vice versa. Whereas clinical trial management system is for managing the process, finances and key clinical checkpoints that the site needs to maintain for each patient on the trial. It’s a system that comes after what users do with Watson for Clinical Trial Matching. An electronic data capture system is used for submitting data about the trial directly to the trial sponsor throughout the trial. So again, it is used much later in the process than Watson for Clinical Trial Matching.

Q6) How does Watson learn about new cancer types and patient information and how accurate is it?

Building off of Watson’s base of general medical knowledge, it has to learn the specifics about an individual disease to then become smart about the data. To train the system, we use subject matter experts from Mayo Clinic, our training partner. So if we want to teach Watson about a new disease, we will work with subject matter experts at Mayo to decide which characteristics are important for patients with that disease. We then get sample patient data from Mayo so that we can teach Watson about those particular clinical attributes. We go through several iterations of refining Watson’s accuracy with that defined data, applying certain medical logic at certain times so that, over time, when Watson sees various forms of data around that disease it can identify it, derive it and extract it from the unstructured data to create a clinical profile for the patient.

At the same time, we go through a similar learning process to teach Watson about typical clinical trials and eligibility criteria for that disease. Once we’ve done that, we again go through several iterations, so Watson can learn to correctly identify potential trials for patients based on the characteristics it has learned.

Watson’s accuracy is measured from two perspectives using statistical analysis: how good is Watson at finding the correct patient characteristics, and then equally important, how good is Watson at correctly identifying trials that match the patient characteristics. Before a disease is released into production, Watson has to meet a certain threshold of accuracy, as agreed upon by the experts at Mayo Clinic.

Q7) What is the clinical and business value of Watson for Clinical Trial Matching and why is it important to the overall landscape of oncology/healthcare today? 

From a business perspective, the value Watson for Clinical Trial Matching provides to enrollment efforts cannot be overemphasized. Achieving enrollment targets is key, meaning a site has to screen many patients. If a site is participating in a clinical trial, it is committing to attaining its enrollment target and will be operating the trial within a very defined budget. If the site under-enrolls, then the trial is likely to cost them more than what they’re reimbursed for.

Cancer centers want to be viewed as leaders and innovators in the oncology research space. They are committed to advancing research and innovative treatments for patients. That requires successful trials, which requires they meet their enrollment targets. And meeting enrollment targets is difficult to do unless you can screen a large number of patients. Watson for Clinical Trial Matching helps them do just that.

For patients, it means having the opportunity to learn about and consider clinical trial treatment options that they previously may not have even known existed.

Most trials that fail worldwide do so because they are unable to meet either the enrollment target or the enrollment timeline. If trials fail, we’re not going to advance oncology, we’re not going to advance medicine, and potentially life-saving drugs will never get to market. That doesn’t benefit anyone because ultimately, the goal is to provide better outcomes for more patients.