The How Behind Natural Language Processing (NLP)

By and Andy Sitison | 4 minute read | November 4, 2019

The How

In part one of the NLP blog series, I covered what Natural Language Processing (NLP) is and why industries are starting to implement NLP into their organizations. It’s time now to take a deeper dive and understand the HOW behind NLP.

In today’s world, data can come from many sources, in various forms, and in varying degrees of cleanliness. On top of that, different problems need to be discovered, addressed and solved through the modern uses of AI. As textual data continues to grow at an exponential rate every single day, organizations need to have the necessary tools to break down this ever-growing behemoth and make sense of it.

I want to take you through an example of an organization that is using NLP to help brands grow human-led experiences. The company is Share More Stories.

The Story Behind Share More Stories

Sean: Hi Andy, can you tell us about Share More Stories and how the company began?

Andy: Hi Sean! Yes absolutely, so let’s take a step back for a minute and first think through your short-term memory of all the times you saw a digital advertisement in the last 24 hours. Think about the ads, memes, sidebars, and popups; then ask yourself “was this really the design intent?”

We live in a digital economy. Digital’s impact has been profound, but also has unintended consequences. Humans are engulfed in an always on digital world; through user tracking, ad interruption, puppy pictures, and trolling social commentary. We live in a constant state of digitality and it’s having an impact on our productivity, health and happiness. As we become more accustomed to a constant digital world, brands are also losing their influence and ability to build relationships with the people they care about. For companies, whose future will be decided by their digital prowess, it is becoming increasingly difficult for brands to stay relevant. How do companies cut through a proverbial digital veil? This is where we started Share More Stories (SMS).

The team at SMS is on a journey to help companies rethink the way they engage individuals and communities. We help companies build more authentic relationships with their customers and employees. It starts with building engaging communities. We do that through storytelling experiences. Stories have a special place for humans: they connect us, they help frame our thinking, they describe us more deeply than the words we say. At SMS we engage people around a topic, collect stories anonymously and then build insights for future direction.

Sean: How exactly did Share More Stories leverage Watson NLU within your platform?

Andy: To develop insights, we start with NLP techniques to process the long form text of every story we generate. This technique is different than how others are using NLP today. Instead of analyzing larger, more “shallow” datasets, we focus our efforts on going deeper into smaller datasets. We utilize machine learning to get underneath what the stories are telling, including subtle interrelationships that arise among the data and underlying emotional expressions.

We believe our approach delivers a better result than what can be pulled from a tweet or other short-form text, no matter how many you have. For us it is important to get to a more holistic, more deeply human perspective.

Sean: What about Watson NLU technologies helps your cause?

Andy: IBM Watson has been instrumental in the build out of our products and helping us capture results quickly using a pre-trained deep AI network on our smaller datasets. This was especially useful in our early days by allowing us to spend more time on insight development, without worrying about where we would get the training data from, for every test we wanted to run. This made us more consultative for our clients and more agile in solution development.

By leveraging Watson NLU, we’re able to extract key outputs including concepts and entities across a large set of stories. This metadata is helpful for us to determine how we may group stories together or identify specific stories to explore further.

The data we pull from Watson’s suite of NLP technologies, serves as a base upon which we build algorithmic insights. An example is a multi-regression process that generates what we call emotional bundles (EB). EBs allow us to observe how certain feelings, values and needs are impacting others. This data is very important in understanding the complex nature of human beings. With Watson, we can find in stories what we humans can’t always express directly. With the use of Watson, we can show business leaders how to authentically connect with their employees, consumers, and communities around them. We are helping companies reach the right audiences at the right time, by delivering deeply personal content and unravelling human complexities into clear, data-driven relationships of human emotions, needs and values through stories.

Sean: Thanks so much Andy, looking forward to hearing more stories!

Stay tuned for next month’s NLP blog!