June 27, 2016 | Written by: Tanmay Sinha
Share this post:
Author: Tanmay Sinha and Andrew Boyd
Unstructured Data – An Untapped Opportunity
Every day we produce 2.5 quintillion bytes of new data – the majority of which is unstructured. While this data holds many possibilities, it is vastly underutilized because of the complexity and challenges in using this data. In fact, typical organizations only leverage 8% of this data.
Most critical of this unstructured data is in natural language text e.g. books, journals, health records, e-mails, blogs, tweets etc. Not only are we producing a vast amount of this natural language text daily, but the rate at which we are producing the text is growing – so, there is great opportunity for savvy organizations to leverage this text to make game-changing decisions and provide business value.
How can your organization tap in to all that natural language text for business insights?
Introducing IBM Watson Knowledge Studio
IBM Watson Knowledge Studio is a cloud-based application that enables developers and domain experts to collaborate on the creation of custom annotator components that can be used to identify mentions and relations in unstructured text.
The effort behind IBM Watson Knowledge Studio began almost two years ago when a joint team from IBM Research and the IBM Watson Group did a study to understand existing natural language processing (NLP) technologies and best practices. The goal of the study was to recommend a common direction for tooling, repositories, data formats and adapters to enable interoperability across IBM’s cognitive applications and reduce the time needed to customize them.
Knowledge Studio has come a long way in the past couple of years, but our founding vision to establish interoperability and simplify domain adaptation to create cognitive applications has remained constant.
The traditional approach to adapt, or train, a cognitive application requires the subject matter expert to know NLP concepts, such as understanding syntax for creating rules. But not anymore. With Knowledge Studio, we are reinventing how you can teach Watson and create cognitive applications specifically for your domain.
Understanding Natural Language Text
Systems like IBM Watson provide Natural Language Processing power to “understand” unstructured text. The problem, however, is that most organizations are looking to mine unstructured text for specific information and context that are unique to their industry or business needs. As a result, organizations must customize the NLP model in order to realize the full value/benefit of mining the unstructured text.
For instance, imagine you are an auto manufacturer; you’re looking to proactively identify safety defects using traffic incident reports. To achieve this, you would need to create a NLP model that – through context – understands manufacturer, make, model, type of incident and date of incident.
While the example is straight-forward, the process of customizing the NLP model can be a cumbersome process that requires a rate set of specific skills. The good news is that it doesn’t have to be.
IBM Watson Knowledge Studio dramatically lowers the bar for customization of natural language processing models – what’s more, through innovations in underlying technology, the process is much faster than traditional approaches.
Watson Knowledge Studio is:
- Intuitive: Use a guided experience to teach Watson nuances of natural language without writing a single line of code
- Collaborative: SMEs work together to infuse domain knowledge in cognitive applications
- Cost effective: Create and deploy domain knowledge infused annotators faster than ever before using an integrated development environment
Unlike tools from the past, Knowledge Studio is built from the ground up for subject matter expert and developers to collaborate. It leverages decades of IBM experience in NLP and applies the IBM Design Thinking approach to distill the training process down to the most user-friendly experience possible.
The various components of Knowledge Studio come together to address all aspects of domain adaptation for unstructured text – starting from defining a type system to deployment of a fully trained model directly to offerings IBM Watson Explorer and AlchemyLanguage.
IBM Watson Knowledge Studio and AlchemyLanguage – Custom Real-time NLP on Cloud
Since AlchemyAPI joined IBM Watson, we’ve continued to enhance the capabilities of our platform.
As part of that effort, we are excited to announce that you can now “train” a custom model about your industry or specific knowledge domain using IBM Watson Knowledge Studio. Here is how you can to get started creating and using your custom model:
- Purchase Watson Knowledge Studio
- Get your Alchemy API key from Bluemix – create your key using the Advanced plan
Train and publish a model
- Use your model through the API
IBM Watson Knowledge Studio and Watson Explorer – All enterprise text analytics needs addressed in an integrated solution
Watson Explorer is a technology platform that accesses and analyzes structured and unstructured content. Explorer presents data, analytics and cognitive insights in a single view and gives customers the information they’re looking for while uncovering trends, patterns and relationships.
With the release of Watson Explorer 11.0.1, customers can utilize machine learning models developed using Watson Knowledge Studio to annotate and enrich content on premise. Watson Explorer is the first and only IBM offering to support a hybrid approach that allows both custom rules and custom ML annotators for annotating unstructured text.
Try IBM Watson Knowledge Studio Today
IBM Watson Knowledge Studio dramatically lowers the time and expertise needed to train a Watson solution for a particular domain or area of expertise – what was once measured in years is now measured in weeks. Most importantly though, you are teaching Watson, not programming it.
IBM Watson Knowledge Studio represents an innovative new approach to training natural language processing models and opens up new possibilities for cognitive computing across industries.
Questions? We are always here to help. Reach out to us via our support forums.
IBM Watson Knowledge Studio is available on IBM Marketplace.