August 12, 2019 | Written by: Oktie Hassanzadeh
Share this post:
The extraction of cause-effect relationships from text is an important task in knowledge discovery with numerous applications in medicine, finance, scientific discovery, and risk management. At the 28th International Joint Conference on Artificial Intelligence (IJCAI’19), we present new results on causal knowledge extraction from large amounts of text for applications in enterprise risk management in our paper, “Answering Binary Causal Questions Through Large-Scale Text Mining: An Evaluation Using Cause-Effect Pairs from Human Experts.”
The system employs a novel approach for extracting causal statements using state-of-the-art neural network-based techniques to provide an unsupervised method of answering questions in the form “Could X cause Y?,” where X and Y are phrases without any semantic constraints.
This extracted causal knowledge is used to build a domain model for scenario planning and decision support. This work is inspired by a core problem in our work on Scenario Planning, where users need to provide the system with a causal “mind map” — a graphical representation of important “drivers” (events or conditions) and potentially causal links between them. This is a complex task that requires deep expertise in the target domain and applications and many iterations involving hours of experts’ time.
A screenshot showing the outcome of scenario planning in IBM Scenario Planning Advisor project. The indicators/risks/implications are “drivers” (events or conditions) that are manually defined and linked, a tedious task that our work aims to automate.
Our causal knowledge extraction system is capable of ingesting a large corpus of text, e.g. a corpus of 180 million news articles, extracting causal statements, and building index structures to effectively support automated question answering.
We evaluated three classes of question answering methods:
1) Methods that rely on statistical analysis of the full corpus, used as baseline methods.
2) Methods that rely on extraction of cause-effect pairs from sentences in the input corpus and building indices for querying them. In particular, our “DCC-Embed” method uses a semantic index of all phrases in the input corpus. This index is built using a neural network-based embeddings model trained over the input corpus.
3) Methods that rely on a semantic index of causal sentences.
The “NLM-BERT” method uses the newly popular Bidirectional Encoder Representations from Transformers (BERT) Neural Language Model to turn the sentences into vector representations to enable semantic querying. Our DCC-Embed and NLM-BERT methods showed very promising results in our evaluation.
Our “DCC-Embed” method results over the results reported by [Sharp et al., 2016]: R. Sharp, M. Surdeanu, P. Jansen, P. Clark, and M. Hammond. Creating Causal Embeddings for Question Answering with Minimal Supervision. In EMNLP, 2016.
We faced a major challenge in evaluation of our binary causal question answering methods as the existing benchmarks had several limitations and did not address the requirements of our target applications in Enterprise Risk Management. As a result, we developed three novel benchmark data sets of cause-effect pairs and released them publicly (https://doi.org/10.5281/zenodo.3214925
). We hope that these data sets will promote further research on causal knowledge extraction and binary causal question answering methods.
We intend to use the outcome of this work as a part of the IBM Scenario Planning Advisor [Sohrabi et al., 2018 , 2019  to enable users to prune a large space of potential pairs, and to provide hints for each question posed to the user.
- Sohrabi, A. Riabov, M. Katz and O. Udrea. An AI Planning Solution to Scenario Generation for Enterprise Risk Management. Proceedings of the 32nd Conference on Artificial Intelligence (AAAI-18), 2018
- Sohrabi, M. Katz, O. Hassanzadeh, O. Udrea, M. D. Feblowitz, and A. Riabov. IBM scenario planning advisor: Plan recognition as AI planning in practice. AI Commun., 32(1):1–13, 2019.