With the help of machine learning-based natural language processing (NLP) models, your business can speed up processes, make more accurate predictions, and uncover new insights from your existing data. To get the maximum value for your efforts, it is important to choose the right machine learning-based NLP service for your specific use case. At IBM, we often get questions like, “How does a machine learning model know how to read my data?” or “How does a machine learning model know what to look for?” The answer depends on which model you use.

Let’s discuss two different approaches to building machine learning solutions with NLP: out-of-the-box models and custom models. When you need to quickly uncover general insights, you probably want an out-of-the-box model. When your use case contains specific industry or domain data, such as specialized data or jargon, a custom model is appropriate.

Modeling in NLP

Out-of-the-box models

We frequently hear that building solutions for text-based analysis is cumbersome and time-consuming. Spending hours sorting through documents, labeling keywords or sentiment in the text, isn’t the best use of your time. Out-of-the-box or pretrained NLP models automate those tasks for you. Out-of-the-box models provide tremendous advantage when you have limited time or don’t have data to train on. They’re effective for quick proofs of concept and have a high ROI, especially when you need to understand higher level items like categories or concepts within your documents. Prebuilt models are often trained on general purpose data sources and tend not to be tailored to any one industry or domain.

Custom models
When your use case contains specific industry lingo, pretrained models may only meet a portion of your needs. Across different industries, some entities (people, places, events, etc.) will have different meanings, so an out-of-the-box model may not be able to pick up on these nuances depending on the context. For example, the word “premium” can mean the top end of a product range, or it can mean the payment made in exchange for an insurance policy. In Australia, this insurance payment is referred to as “the cover”.

Your custom model can be trained to understand the underlying nuances, meaning, and relationships specific to your use case and industry domain. As above, you can teach the model what the word “premium” means in the context of your business. These models can be built in a few ways, and you don’t have to be a data scientist or a software developer to create them. With code-free tooling, all you need to bring is subject matter expertise. A subject matter expert (SME) will define what information matters in the model, upload documents for training, and then select examples, often called annotations, within those documents.

The notations you create on your documents provide the system with specific examples that serve as the “ground truth” for the model. Whether defining specific entities for extraction from a document or the relationships between them, through training, you’ll make the improvements which ensure the model’s effectiveness.

When should you invest in a custom model?

When combining out-of-the-box NLP models with custom NLP models, you get the best of both worlds. We recommend training your model to understand simple components first, such as locations or organizations. This way, your NLP model homes in on the pieces that matter to your business, to ultimately drive smarter decisions.

Now what?

Learn more about the differences between out-of-the-box NLP models and custom NLP models, or check out Watson Natural Language Understanding and Watson Knowledge Studio to build your very own machine learning-based NLP models.

Was this article helpful?

More from Artificial intelligence

10 tasks I wish AI could perform for financial planning and analysis professionals

4 min read - It’s no secret that artificial intelligence (AI) transforms the way we work in financial planning and analysis (FP&A). It is already happening to a degree, but we could easily dream of many more things that AI could do for us. Most FP&A professionals are consumed with manual work that detracts from their ability to add value to their work. This often leaves chief financial officers and business leaders frustrated with the return on investment from their FP&A team. However, AI…

ServiceNow and IBM revolutionize talent development with AI

4 min read - Generative AI is fundamentally changing the world of work by redefining the skills and jobs needed for the future. In fact, recent research from ServiceNow and Pearson found that an additional 1.76 million tech workers will be needed by 2028 in the US alone.  However, according to the IBM Institute for Business Value, less than half of CEOs surveyed (44%) have assessed the potential impact of generative AI on their workforces. To help customers develop and upskill their workforces to meet…

Responsible AI is a competitive advantage

3 min read - In the era of generative AI, the promise of the technology grows daily as organizations unlock its new possibilities. However, the true measure of AI’s advancement goes beyond its technical capabilities. It’s about how technology is harnessed to reflect collective values and create a world where innovation benefits everyone, not just a privileged few. Prioritizing trust and safety while scaling artificial intelligence (AI) with governance is paramount to realizing the full benefits of this technology. It is becoming clear that…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters