Here’s how Watson Assistant and Soul Machines are helping the UN keep the oceans clean

By | 4 minute read | September 15, 2020

The face mask you’re wearing or the plastic bottle you’re drinking from can easily find its way into the world’s oceans — through improper disposal, or through flaws in the waste management chain like improperly managed landfill sites—and join the 5 trillion pieces of detritus clogging our oceans. The trash that traps and drowns birds and aquatic mammals can also be ingested by marine animals, causing them to choke or starve. This deluge of waste kills over 100,000 marine mammals a year and contributes to damage to coral reefs worldwide, further threatening ocean life and the buffer that protects the shoreline, and human lives, from high waves and destructive storms.

In 2015, the United Nations Environment Programme (UNEP) rallied marine experts, environmentalists, nonprofits, academics, and citizen scientists worldwide to broadly confront conservation and sustainable living and significantly reduce marine pollution by 2025. The massive amount of marine litter is mirrored by the vast amount of data collected to understand the problem, gauging things like litter density and location. However, that data has not been centralized or made easily accessible.

Seeing an opportunity to organize and use this data and change the world for the better, the IBM Data Science and AI Elite team (DSE) collaborated with UNEP and the Wilson Center to understand how to best make an impact. That impact comes in the form of a virtual expert named Sam. Powered by IBM Watson Assistant (a conversation AI platform) and Soul Machines (a world leader in humanizing AI and creator of Digital People) Sam is an autonomously animated, emotionally responsive virtual agent with a human-like avatar.

After gathering marine pollution data, the DSE knew that having that data and making it accessible still doesn’t fully solve the problem. Humans tend to become passionately engaged in finding solutions when they make an emotional connection to a problem and their part in its cause and solution. So, IBM and Soul Machines set out to create a virtual expert with which users will feel comfortable engaging.

Powered by Soul Machines technology, Sam can “see” and react to the user’s emotional state; his virtual nervous system interprets the situation and engages appropriately. While the Soul Machines technology makes these interactions feel natural, the other half of the problem is to ensure that the artificial humans can answer user questions correctly and accurately, with a deep understanding of spoken language and the topic at hand. That’s where IBM Watson Assistant comes in.

Watson Assistant is an enterprise-level AI assistant — customizable to any business — that delivers proactive, personalized services. It interprets natural language (in multiple languages) and can be continuously trained on domain-specific data to provide appropriate responses to user queries. It is a massively scalable alternative to human customer service agents, with the ability to automatically handle thousands of spoken language queries.

“Many of our clients were already working with IBM, using Watson-based chatbots to answer customer questions through text-based interfaces,” says Greg Cross, Chief Business Officer at Soul Machines. “We just needed to integrate our artificial humans with Watson Assistant to provide a new user experience layer that would deliver Watson’s answers as if you were talking to someone live and in-person.”

As a user speaks to Sam, perhaps inquiring specifically about clean-up efforts around marine litter, Soul Machines sends the audio stream of the customer’s voice to Watson. Using Watson Assistant and speech-to-text, Watson converts the audio into text, then searches the repository of knowledge for relevant answers to the question, ranks the results, and returns the top-ranked response to the Soul Machines solution.

Meanwhile, the Soul Machines platform analyzes the audiovisual input for emotional cues from the customer’s tone of voice and facial expressions. It then converts the answer into modulated, emotionally inflected speech for Sam to deliver, matched with appropriately generated facial expressions.

Nicholas Holmes, IBM’s global government chief technology officer for data and artificial intelligence, says, “Sam was developed to create an emotional bond between the issue of marine litter and the user, because we believe that emotional bond will lead to action.”

Watson Assistant interprets a user’s intent and retrieves relevant information from UNEP’s vast repository and other sources.

“Our goal for this proof of concept was to show that IT, and more specifically, AI solutions, both have a critical role to play in tackling the issue of marine litter and helping the United Nations meet the Sustainable Development Goals,” Holmes adds.

Learn more about what happened when the IBM Data Science and AI Elite team (DSE) joined forces with the UNEP and the Wilson Center to sort through oceans of data to understand marine litter fully. And meet Sam, the virtual environmental advocate designed to unify researchers, communities, and policymakers.

Grab your front row spot today for IBM’s Data and AI Virtual Forum on September 16 and prepare to dive deep into the story and learn how IBM handled three massive data challenges in the world’s fight against marine litter.