Max Kelsen

Detecting citizen concerns with 97% accuracy using IBM Watson

The company selected IBM® Watson® Discovery offerings to build an insight engine for powering a new AI platform that can deliver highly accurate, detailed insights into customer experiences. Now, it can more effectively arm businesses and governments with the specific knowledge needed for taking targeted actions to solve problems.

Business Challenge

To help organizations and businesses improve their customer experience, Max Kelsen sought to unlock insights hidden in unstructured data, including social media, call center logs, letters and emails.


The company selected IBM Watson Discovery offerings to build an insight engine for powering a new AI platform that can deliver highly accurate, detailed insights into customer experiences.


97% accuracy achieved 
in new insights that a local government entity used to drive ROI on initiatives to improve citizen experiences

≈2 weeks to create AI models
using Watson tools that save the startup time and money by not requiring coding 

Delivers new business value
to organizations by extracting precise insights from their unexplored customer data

Business challenge story

Seeking new depths of analysis
Max Kelsen helps companies explore the previously unexplorable—unstructured qualitative customer data—to gain unprecedented insights for driving higher ROI on customer-experience initiatives. Passionate about AI, the people who founded and grew the company recognized that traditional means of data gathering and analysis couldn’t provide the depth of detail required to identify specific problem areas.

“A lot of companies hit a roadblock on what they can improve using traditional approaches,” explains Nicholas Therkelsen-Terry, Chief Executive Officer (CEO) and Cofounder of Max Kelsen. “Many come to us and say, ‘We’ve done focus groups and surveys, but we’re not sure where to go from here to get better outcomes.’ Max Kelsen comes in and helps diagnose the problem.

“We look across their omnichannel world to understand where and when customers are being let down by the experience with that brand and identify exactly why it’s happening, when it’s happening and to whom,” he continues. “Then, they can put in highly targeted programs to improve the customer experience and move the metrics up.”

Max Kelsen refers to unstructured customer-experience data as “dark data,” because it lies dormant in call center complaint logs, online surveys, and verbatim comments made in emails and letters and on social media. This untapped resource can reveal a fuller, more precise picture of the customer experience by providing invaluable insights into qualitative, individualized dimensions of relationships, including emotion and sentiment.

Typically, governments and businesses gauge customer satisfaction by analyzing structured data, including demographic and transaction information gathered from CRM and other systems, product ratings and loyalty metrics. Yet aggregated, quantitative measurements extrapolated from this type of data generally provide an incomplete picture. They fail to accurately decipher the customer’s, or citizen’s, voice at each stage of the journey, making it hard to precisely identify the issues that need addressing. As a result, organizations can neither invest wisely nor take the proper steps to repair and strengthen relationships.

For example, many call centers use customer satisfaction surveys to help generate Net Promoter Score (NPS) loyalty metrics. A person might rate his or her overall call center experience as a 5 on a scale of 1 (poor) to 5 (excellent), because the representative adequately addressed the core issue. However, that person’s written comments might indicate that the representative was unfriendly, feedback that remains unheard.

Therkelsen-Terry explains that many large organizations already have huge volumes of unstructured data potentially full of business value but unexaminable without cognitive tools. They often spend millions of dollars engaging outside market research firms to help them collect and analyze new qualitative data. But despite the investment, these organizations still can’t tap in to the nuanced, individualized aspects of their customers’ experiences.

Max Kelsen seeks to offer a more cost-efficient, exact approach by using AI technologies to help organizations take advantage of the qualitative data underlying their more typically analyzed quantitative data. The company began by analyzing Twitter and other public social media data. However, as the business expanded, Max Kelsen started looking at ways to intelligently search unstructured private data, regardless of source, format or length, to reveal the nuance and context of myriad customer interactions.

At one point, a local market research firm approached Max Kelsen on behalf of one of its customers, a government entity in Australia, seeking detailed insights into issues affecting the safety and satisfaction of millions of its urban residents. Citizen feedback collected through surveys and other traditional means, for example, revealed high levels of anger and frustration concerning cycling. Government leaders set aside funds to build more bike lanes in popular cycling spots; however, too many questions regarding the nature of the problem remained unanswered. Exactly who was upset and why? Were cyclists frustrated with a lack of infrastructure, or were drivers angry about sharing city streets? Or were other groups and factors at play? Before they could confidently invest taxpayer funds to develop more infrastructure, officials sought clear, accurate answers to help guide their decisions.

To deliver the deeper insights the government and other enterprises requested, Max Kelsen accelerated development of its intelligent platform. It needed advanced AI tools, including natural language processing (NLP) technologies for uncovering emotions, sentiments and relationships. Given the young organization’s limited resources, it also sought to quickly and efficiently build and train customized machine learning models.

The depths of analysis we’re able to achieve with our Watson tooling is far beyond what we were able to achieve beforehand.

Nicholas Therkelsen-Terry, Chief Executive Officer and Cofounder, Max Kelsen

Transformation story

Uncovering insights with IBM
Max Kelsen chose an IBM Watson Discovery solution to build an insight engine for powering its new offering, the cognitiveCX platform. The accuracy of the new business knowledge revealed using the Watson toolset, which combines cloud-based cognitive search, machine learning, relevancy training, NLP and query capabilities, propelled its decision. “We tested a number of commercial and open source solutions, and Watson consistently outperformed them,” says Samuel Irvine Casey, Chief Operating Officer (COO) and Cofounder at Max Kelsen.

Analysts’ ability to quickly build and train highly accurate customized models using IBM Watson Knowledge Studio , which does not require coding, also stood out. “No solutions on the market allowed us to train a custom model for any domain as easily or effectively as WKS [Watson Knowledge Studio],” notes Irvine Casey.

Therkelsen-Terry agrees. “For us, that was game-changing. It brought a lot of what we were thinking alive and allowed us to execute the way that we wanted to. That was really the moment for us, a light bulb moment, when we first got onto WKS and said, Wow, this is insane.”

Irvine Casey and his team in the Insights and Analytics department collaborated with the government entity and Sherlok, the human analytics services division of the marketing research firm, to build and test the Watson insight engine on the IBM Cloud™ platform. The government organization, which like many businesses receives feedback from multiple channels and departments, presented an ideal use case for identifying the real pain points preventing delivery of excellent experiences.

Max Kelsen began its extensive analysis using Watson Knowledge Studio, a cloud-based solution that facilitates annotation of custom, industry-specific components in unstructured data. Government staff provided approximately two million source documents collected from hundreds of citizen touchpoints dating back three years and from more than 40 distinct service areas. The Max Kelsen team randomly selected 5,000 of the documents for annotation. In preparation for working with Watson Knowledge Studio, it mapped out relevant entity types, including service departments; web, social and call center channels; and citizens’ emails and letters to officials. Then, without coding, the team trained a machine learning model using approximately 4,000 of the annotated documents, which provided the equivalent of the 300,000 words of content IBM recommends for analysis.

Within six weeks of receiving the source documents, Max Kelsen sent the government officials a preliminary report of the analysis results. During that period, the team became acquainted with the application’s user-friendly features for training rule-based and machine-based models.

“When we first started,” says Therkelsen-Terry, “we didn’t grasp the power of some features—like preannotation tools, dictionaries and a visual rule editor—that we now use instinctively to really speed up the training process. Done manually, using 300,000 words, this would be an incredibly arduous task.” For instance, the team relied heavily on the application’s human annotator feature to help ensure the model accurately and consistently represented organizational and industry terminology.

Having completed the custom annotation model, the team generated qualitative insights using IBM Watson Natural Language Understanding (US). Now embedded with IBM Watson Discovery, the API interrogates unstructured text on an entity-by-entity basis for tone, intent, sentiment, service area and contextual details. The team analyzed the results using the Watson Discovery search engine, which provides core AI capabilities for quickly and affordably uploading, enriching, indexing and querying large collections of private and public data.

“Our analysts can quickly search through millions or billions of records in an instant to get individual outputs out of what customers are saying about certain issues and why they’re frustrated,” elaborates Therkelsen-Terry. “We can really get a handle on how customers are feeling, what they’re talking about and the actual words they use.”

Max Kelsen also relied on leading data visualization tools to present the insights in meaningful, engaging ways.

Results story

Helping drive ROI
Using Watson tools, Max Kelsen built and trained an AI model for the local government entity that delivers insights with 97 percent accuracy. And because the model’s algorithms can learn from the patterns it detects in new data, its accuracy can improve over time.

Max Kelsen built this first model within four weeks, a process the company can complete in as few as two weeks now that the team is familiar with the toolset. Accuracy levels can vary based on variances in the data being used but are still exceptionally high, according to Irvine Casey. “There is no coding involved, which is just a massive plus for our team, because our analysts are here to analyze data and write reports, not to write code. And whether it’s in the telco, financial or government sector, we can quickly train a custom model with an incredibly high accuracy rate,” he explains.

The solution also minimizes demands placed on customers, who don’t need to fund new surveys but only provide knowledge experts to help with annotation.

Now, Max Kelsen can confidently move forward to help enterprises worldwide discover new opportunities to enhance customer experiences and boost ROI using nuggets of knowledge once buried in their data.

“The depths of analysis we’re able to achieve with Watson are far beyond what we were able to achieve beforehand,” says Therkelsen-Terry.

The myriad observations the company provided the government entity offer compelling examples. For instance, officials gained a new understanding of issues related to citizen aggravation associated with cycling. They learned that most complaints originated with pedestrians rather than drivers or cyclists. People walking along the urban area’s many shared bike and pedestrian lanes often felt intimated by cyclists who traveled at high speeds and sometimes even injured them.

Armed with this insight, government leaders avoided investing taxpayer dollars in new infrastructure, which would not have resolved the core issue. Instead, they took other, less expensive actions that directly addressed the problem, thereby saving public funds and improving citizen safety.

“The insight changed the way people thought about a problem and how to address it,” comments Irvine Casey.

In another example, Max Kelsen mapped complaints regarding waste management services from a handful of residents living on a few streets. Traditional surveys alerted officials to poor performance in this area but didn’t tell them where the fault originated, whether with broken bins or a lack of follow-up. Max Kelsen detected the anger because its analysis targeted emotion and sentiment in data sets previously siloed across service areas and channels. In response, the government transformed the processes it uses to manage and rapidly respond to citizen concerns, no matter how isolated, regarding pickup services.

In recognition of their groundbreaking work with the government entity using the cognitiveCX platform, Max Kelsen and its partner Sherlok won a BigInsights Data Innovation Award in 2017. A leading advisory and research firm focused on data analytics, BigInsights bestowed Max Kelsen and Sherlok with awards in the Best Industry Application of AI/Cognitive and Best Customer Insights categories for the government sector.

“We can truly hear the customer voice,” notes Therkelsen-Terry. “We can tell when people are upset, which service disappointed them, and why.”

“That’s something that just wouldn’t have been possible without Watson,” adds Irvine Casey.

About Max Kelsen

Part of a vibrant startup community in Brisbane, Australia, Max Kelsen is an analytics and software engineering agency specializing in applying machine learning and AI technologies to drive innovation. The company employs data scientists, research analysts and software engineers who help enterprises uncover rich, actionable insights from undertapped internal and public data sets, including social media. Founded in 2015, Max Kelsen builds strong collaborative relationships with businesses and organizations in financial services, health care, government and other industries.

Take the next step

To learn more about the IBM solutions featured in this story, please contact your IBM representative or IBM Business Partner, or visit the following websites:

Increase productivity with Watson (US).

© Copyright IBM Corporation 2018.
IBM Corporation, IBM Watson, New Orchard Road, Armonk, NY 10504.

Produced in the United States of America, September 2018.

IBM, the IBM logo,, IBM Cloud and IBM Watson are trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the web at “Copyright and trademark information” at (US).

This document is current as of the initial date of publication and may be changed by IBM at any time. Not all offerings are available in every country in which IBM operates.

The performance data and client examples cited are presented for illustrative purposes only. Actual performance results may vary depending on specific configurations and operating conditions. THE INFORMATION IN THIS DOCUMENT IS PROVIDED “AS IS” WITHOUT ANY WARRANTY, EXPRESS OR IMPLIED, INCLUDING WITHOUT ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND ANY WARRANTY OR CONDITION OF NON-INFRINGEMENT. IBM products are warranted according to the terms and conditions of the agreements under which they are provided. The cognitiveCX platform is not an IBM product or offering. The cognitiveCX platform is sold or licensed, as the case may be, to users under Max Kelsen’s terms and conditions, which are provided with the product or offering. Availability, and any and all warranties, services and support for the cognitiveCX platform, is the direct responsibility of, and is provided directly to users by, Max Kelsen.