Watson

IBM Watson just got more accurate at detecting emotions

Share this post:

emotion_analysis

Emotion detection has been a central piece of the puzzle to make AI systems compassionate. With this goal in mind, early this year IBM Watson released textual emotion detection as a new functionality within the Alchemy Language Service and Tone Analyzer on the Watson developer cloud.

We are pleased to announce that IBM Watson’s emotion detection capability has undergone significant enhancements. These new enhancements were built on the ensemble framework described in the previous article. These enhancements will remain pivotal in improving user interactions, and understanding their emotional state.

What are the new enhancements?

Newly released emotion model brings following enhancements:

  • Expansion in the training data: We doubled our training dataset from the previous release. Systematic expansion of the training dataset has helped the new model to significantly improve its vocabulary coverage than before.
  • New feature selection process: Feature selection is one of the most important steps in building a large scale machine learning system. In this release, we explore some linear models penalized with the L1 norm to have coefficients of important features to be non-zero. Based on our experiments, we find that Linear SVM with L1 penalty helped most to extract important features. These selected features along with topic and specialized engineered features helped classifiers in the ensemble model not only to improve accuracy but also to provide transparency for the final prediction.
  • Diverse classifiers: The ensemble framework performs better when it contains diverse set of classifiers in it. In this release we bring a new set of diverse classifiers exploring different hypotheses, including tree-based ensemble classifiers, kernel-based classifiers, and latent topic-based classifiers. Since training data is continuously increasing, this diverse set of classifiers has to address the scalability problem before being incorporated into our ensemble framework.
  • Improved lexicon support: Our new release significantly improved emotion detection at lexicon/word-level.
  • Expanded support for emoticons, emojis and slang: This is an important step for detecting emotions in conversational systems.

All of these enhancements helped us achieve improved accuracy (in terms of average F1-measure), which is better than the state of the art emotion models [Li et. Al 2009, Kim et.al 2010, Liu 2012, Agrawal and An 2012, Wang and Pal 2015] included in our previous version. Some of these state-of-the art emotion models are part of our ensemble framework.

This is the current state of our work at the time of this release. We are continuously improving our models and look forward to releasing enhanced models in the future.

Ready to try a demo?

Check out these fun (and possibly insightful) service demonstrations:

The API is currently available for English text input. More details about this service, the science behind it, how to use the APIs, and example applications are available in the documentation for AlchemyLanguage and Tone Analyzer.


References

  • Sunghwan Mac Kim, Alessandro Valitutti, and Rafael A. Calvo. “Evaluation of unsupervised emotion models to textual affect recognition.” Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text. Association for Computational Linguistics, 2010.
  • AmeetaAgrawal, and Aijun An. “Unsupervised emotion detection from text using semantic and syntactic relations.” Web Intelligence and Intelligent Agent Technology (WI-IAT), 2012 IEEE/WIC/ACM International Conferences on. Vol. 1. IEEE, 2012.
  • Tao Li, Yi Zhang, and VikasSindhwani. “A non-negative matrix tri-factorization approach to sentiment classification with lexical prior knowledge.” Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 1-Volume 1. Association for Computational Linguistics, 2009.
  • Yichen Wang, and Aditya Pal. “Detecting emotions in social media: A constrained optimization approach.” Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence (IJCAI 2015). 2015.
  • Bing Liu. “Sentiment analysis and opinion mining.” Synthesis lectures on human language technologies 5.1 (2012): 1-167.

Technical team

The technical team responsible for emotion analysis includes: Pritam Gundecha, Hau-wen Chang, Mateo Nicolas Bengualid, Vibha Sinha, Jalal Mahmud, Rama Akkiraju, Jonathan Herzig, Michal Shmueli-Scheuer, and David Konopnicki. Alexis Plair and Tanmay Sinha are the offering managers. Steffi Diamond is the release manager.

Add Comment
One Comment

Leave a Reply

Your email address will not be published.Required fields are marked *


Aftab Hassan

Nice post! Would it be possible to detect sarcasm?

Reply
More What's New Stories

IBM Cloud Container Service now available for IBM Cloud Dedicated

IBM Cloud announced the general availability of Kubernetes support for IBM Cloud Container Service in May, 2017. We are now thrilled to bring this capability to IBM Cloud Dedicated customers. Customers using IBM Cloud Dedicated can now deploy Kubernetes clusters in their environments within minutes.

Continue reading

What’s included in the IBM Cloud Developer Tools CLI version 1.1.0

New features and fixes in version 1.1.0 of the IBM Cloud Developer Tools CLI.

Continue reading

Integrate and Analyze Diagnostic Logs with IBM Cloud Log Analysis

Analyzing diagnostic logs, monitoring application health and keeping track of security-related events are at the foundation of successfully running apps and services. IBM Cloud offers services for that purpose. Today, I am going to show you how to use IBM Cloud Log Analysis to integrate, search and analyze as well as visualize diagnostic logs in the IBM Cloud.

Continue reading