Businesses today have a lot of data—data in the cloud, data on premises, unstructured data, structured data. It’s not uncommon for enterprise AI training plans to contain terabytes of documents that must be ingested, annotated and the enrichments tested before useful results can be delivered. Search processes like that can consume a lot of resources before beginning to generate value. If only there was a way to make search intelligent.
Introducing, smart document understanding (SDU), a powerful feature now available in Watson Discovery, that can give you accurate answers faster, from 10 days to 2 minutes for one leading bank.
Building on advances from IBM Research and drawing on the corpus conversion service released in late 2018, smart document understanding allows you to visually train AI to understand your documents. SDU distinguishes text elements and extracts the most valuable content such as key paragraphs, while excluding noise like footers and headers and even identifying text in images. In addition to tackling some of the most challenging types of documents, Watson’s new intelligent search feature allows you to achieve machine learning in less time with accurate results by training only a small set of pages in your documents.
Find answers to highly specific questions, faster
Consider the time savings – not to mention potential revenue and risk reduction opportunities for a financial institution like U.S. Bank. For businesses today, accepting credit and debit card payments has never been easier. Pricing, on the other hand, remains highly complex involving multiple parties with varying fee structures based on risk, fraud potential, and card-specific rewards. Additionally, pricing can be customized based on the type of customer(s) a merchant sells to. As a result, monthly billing statements breaking down costs and fees is confusing, time consuming, and often times frustrating for both merchants and incoming sales reps hoping to win new business.
To solve this problem, U.S. Bank’s Innovation Team partnered with Elavon, a U.S. Bank subsidiary offering merchant processing and payment solutions to more than 1,000,000 merchant locations around the world. Together they piloted and tested a statement analysis solution capable of analyzing a prospect billing statement in real-time and generating an optimized pricing proposal. Using IBM Watson Discovery, with the enhanced smart document understanding capability, what once took 10 days now takes 2 minutes. The solution can analyze dense documents, often dozens of pages long with thousands of relevant pricing codes and other details, and parse relevant information for merchants and sales reps quickly and clearly. As a result, both merchants and sales reps can now spend time doing what they do best, serving their customers and building meaningful, valuable relationships.
How it works
Smart document understanding is simple to use within Watson Discovery. No technical training is needed. The visual interface allows you to point and click document elements like titles, subtitles, headers, or footers to help Watson distinguish these different types of text from other text in the corpus. Without SDU, a user would spend considerable time manually selecting such text elements and tagging them. Watson learns from your point and click activity and automates that tagging step.
At any point in the process you can submit the training document to Watson, and it will visually display how it’s understanding the fields in your document. If it doesn’t consistently discern footers, for example, from body text, you simply point and click to highlight the footer again and resubmit. SDU is designed to help you train Watson on your documents using a handful of documents, even for very large data sets.
This same point and click classification system can be applied to images, spreadsheets, PDFs, even OCR content.
Small data and UI
You hear a lot about big data, but it’s the small data sets that can drive real business applications for AI. Smart document understanding helps firms focus their AI solutions on the precise data that’s valuable, even if it’s contained in documents that hold lots of un-valuable data.
Small data is particular to an individual organization or a specific industry. It’s narrower and deeper than “big” data.
For example, an oil and gas company’s engineers may have years of historical knowledge – everything from case files, incident reports on a particular rig, geological survey data – but it’s currently siloed with individuals or within separate systems.
A key business advantage lies in tapping into organizational insights, historical customer data, internal reporting, past transactions and client interactions. These elements are too often underutilized.
With Watson, which learns especially well from small data, enterprises can finally gain access to and derive unprecedented insights from this dark data. Plus, insights from small data can be coupled with insights from larger public data sets to give organizations the competitive intelligence needed to set themselves apart.
SDU allows you to tell your AI which data in your documents matters, immediately making your large data set smaller. Like a pair of noise-cancelling headphones Watson search technology can keep your results free from distracting elements and focus only on what you need to know to solve a particular business challenge.
Get cleaner answers with less effort
AI is a powerful way to turn unstructured data into opportunities, cost savings, and risk mitigation, however training AI remains a critical and often time-consuming step in any AI strategy and implementation. Even advanced users of AI constantly seek new training options that can produce results that are specific, personalized to a user’s needs, and highly accurate with less effort and greater speed. Smart document understanding in Watson Discovery provides exactly that advantage.
Let Watson do more of the redundant, time-consuming tasks of the training process so your technical and business experts can spend their time adding greater value that’s only available with their expertise.
To learn more about Watson Discovery’s pricing and features check out the product page.
In my posts over the past few months, I’ve explained what NLP is, why it’s the backbone of AI integration in industries today, and how NLP can be put to work inside organizations. As we close out 2019 and look ahead to 2020, I want to address the future of NLP, where I see it ...read more
What are text analytics? Text analytics can help organizations discover patterns in large unstructured data sets. Unstructured data, such as videos, photos and audio accounts for at least 80% of your company’s data, a true “blind spot” for most businesses. Add to this the fact that every second companies around the world add data to ...read more
As a business owner, shielding your company from risk is among your primary concerns. You want to make certain that your decisions are security minded, sound and based on the best available data. So, when a scandal erupts around the longest running global interest rate standard – the London Interbank Offered Rate, or LIBOR – ...read more