July 10, 2020 | Written by: Prasanna Sattigeri and Dennis Wei
Share this post:
The Thirty-Seventh International Conference on Machine Learning will be held virtually in 2020 due to the COVID-19 pandemic. Join IBM Research AI from July 12-18 to learn more about our work. We will present several demos, talks, and papers that cover a wide range of topics including scaling, automating and improving the trustworthiness of AI systems. Following are some key highlights.
Fairness is a critical characteristic of a trustworthy AI system. It is important to scrutinize AI models for their inherent bias while building highly accurate models so that no individual or group is unfairly affected. IBM researchers have been investigating the trade-off between fairness and accuracy. The resulting paper provides new insights on existing methods of fair classification and their fundamental limits.
Measuring fairness at the individual level can be challenging due to difficulty in measuring the similarity between two individuals. To address this problem, a team of researchers at the MIT-IBM Watson AI Lab and University of Michigan have developed a simple method to learn a similarity metric from the data that can be used to measure and enhance individual fairness.
Robust and Explainable AI
Trustworthy AI systems also require a high degree of robustness and explainability. The relationships between these two properties were explored in two papers by IBM and MIT-IBM Watson AI Lab researchers, in collaboration with MIT. The first paper tackles robustness to adversarial attacks, which are intentionally designed to fool a model into diverging from its expected behavior. The researchers showed, theoretically and empirically, that a proper measure of explainability can result in robust classification.
The second paper introduced a method — “invariant rationalization” — to obtain a rationale or explanation as a small subset of the input features. This work, again at the intersection of explainability and robustness, shows that by learning explanations that are robust to changes in environment, models become less susceptible to spurious correlations and explanations are better aligned with human judgements.
The same idea of invariance was studied by IBM researchers to address the reliability of models on unseen test data when the test data distribution differs from the training data. While standard training of AI models can result in overfitting to spurious correlations, the IBM researchers used game theory to find robust predictors that concentrate on features with causal relationships.
Explainable models are generally easier to understand and diagnose. But explainable models can underperform in terms accuracy when compared to complex models. IBM researchers developed an approach to enhance simple explainable models by leveraging the sample hardness information obtained from complex models. This approach of reweighing the training dataset based on sample difficulty level was shown to reduce the performance gap between simple models and complex models.
Automating and Scaling AI
AutoAI refers to the automation and scaling of labor-intensive tasks in an AI system pipeline with the goal of boosting the productivity of model developers. Examples of some tedious tasks are hyper-parameter tuning and model architecture search. A new method developed by IBM researchers can speed up the search for best neural-network model architecture by several folds without performance degradation. By considering the neural architecture search task as a ranking and transfer learning task, the method is able to predict and stop underperforming choices early on, thus saving computational budget.
Another joint collaboration between IBM and MIT researchers described an approach to perform “one-shot” model fusion from pre-trained models, saving communication costs and training time.
Also, with the goal of speeding up training time, a team of IBM and University of Oxford researchers developed quantum techniques and improved the time complexity of AdaBoost — a framework to boost weak models into strong accurate models.
Sponsorship and virtual booth
IBM Research AI is proudly sponsoring ICML 2020 as a Platinum Sponsor, as well as the Women in Machine Learning Workshop. We hope you will join us at our virtual booth to see and hear about our latest technology demos, publications and career opportunities including the AI Residency Program. For a full list of our papers and demos at the conference, see below.
Live Demos at IBM Booth
- COVID-19 Molecule Explorer: We have developed robust generative frameworks to quickly create novel drug candidates for COVID-19, as well as other peptides, proteins, and materials. We are working with several partners on validating the AI-generated molecules by using in-silico simulations and wet lab experiments.
- Command Line AI (CLAI): Explore and interact with the future of the Command Line with CLAI — Command Line AI — an open-source project that brings the latest in AI and ML technologies to the command line as “skills” to make the user’s daily life more efficient and productive.
- Link Prediction for Master Data Management: We present our latest research to predict links between entities in Wikipedia (as an example use case) using graph neural networks along with human understandable explanations for the predicted links. We also present GraphSheets, a checklist for model developers to ensure that link prediction remains fair, privacy preserving, and ethical.
Expo Talk: Auto AI at IBM Research
Sun Jul 7th 7:45 a.m. – 8:45 a.m. PDT
Automated AI/ML makes it easier for data scientists to develop pipelines by searching over hyperparameters, algorithms, data preparation steps, and even pipeline topologies. This talk consists of three parts, each with a demo:
- Lale: Type-Driven Auto-ML with Scikit-Learn (includes a 10 minute demo) — Lale is an open-source library of high-level Python interfaces that simplifies and unifies the syntax of automated ML to be consistent with manual ML, with other automated tools, and with error checks. It also supports advanced features such as topology search and higher-order operators.
- AutoMLPipeline: Symbolic ML Pipeline Composition and Parallel Evaluation (~15 minute demo) — AutoMLPipeline is a Julia toolkit that makes it trivial to create complex ML pipeline structures using simple expressions and evaluate them in parallel. It leverages the built-in macro programming features of Julia to symbolically process and manipulate pipeline expressions.
- AutoAI with Stakeholder Constraints (10 minute demo) — Common applications of AI involve multiple stakeholders with requirements beyond a single objective of predictive performance. This toolkit automatically generates pipelines with favorable predictive performance while satisfying stakeholder constraints related to deployment (inference time and pipeline size) and fairness. It also provides an API to specify custom constraints.
Presenters: Martin Hirzel, Paulito Palmes, Parikshit Ram, Dakuo Wang
Expo Talk: Federated Reinforcement Learning for Financial Portfolio Optimization Using the IBM Federated Learning (IFL) Platform
Sun Jul 12th 6:30 a.m. – 7:30 a.m. PDT
Reinforcement learning is a natural paradigm for automating the design of financial trading policies. Training the trading policies on historical financial data is challenging because financial data is limited to a few values per trading day (e.g. stock daily close price) and, as such, the amount of training data is relatively low.
Federated Learning offers a potential solution by training on many parties’ data, thereby increasing the amount of training data overall. A recent work by this team shows how it is possible to convert an RL strategy for training a portfolio optimization policy on a set of assets to a multi-task learning problem that benefits tremendously from federated learning. We implement the method on the federated reinforcement learning capability of the IBM Federated Learning (IFL) platform.
The session includes three sections:
- We first give a mini-tutorial on using the IBM Federated Learning (IFL) platform for any federated reinforcement learning problem and illustrate it on the openAI gym pendulum example.
- A demo shows how the IFL works on the financial portfolio optimization problem.
- The accompanying talk provides more details on the method and results.
Presenters: Peng Qian Yu, Hifaz Hassan, Laura Wynter
Expo Demos: RXNMapper – AI Explainability 360 – Command Line AI – COVID-19 Molecule Explorer
- RXNMapper: Unsupervised attention-guided atom-mapping Explore the attentions of a Transformer model that has learned to solve the NP hard problem of how atoms rearrange in chemical reactions on its own, with no supervision or human guidance.
- AI Explainability 360 (AIX360): An open-source Python toolkit for explaining data and machine learning models in diverse and state-of-the-art ways to address the needs of different stakeholders. This demo provides a glimpse of its capabilities, algorithms, and industry domains.
- Command Line AI (CLAI): Explore and interact with the future of the Command Line with CLAI, an open-source project from IBM Research that brings the latest in AI and ML technologies to the command line as “skills” and seeks to make the command line-user’s daily life more efficient and productive.
- COVID-19 Molecule Explorer: The traditional drug discovery pipeline is time and cost intensive. To deal with new viral outbreaks and epidemics, such as COVID-19, we need more rapid drug discovery processes. We have developed robust generative frameworks that can overcome the inherent challenges to create novel peptides, proteins, drug candidates, and materials. We are working with several partners on validating the AI-generated molecules by using in-silico simulations and wet lab experiments, and will include those validation results into the exploration tool as they arrive.
Presenters: Ben Hoover, Hendrik Strobelt, Teodoro Laino, Vijay Arya, Amit Dhurandhar, Tathagata Chakraborti, Kartik Talamadupula, Mayank Agarwal, Payel Das, Enara Vijil
Accepted Papers at ICML
Enhancing Simple Models by Exploiting What They Already Know
Amit Dhurandhar, Karthikeyan Shanmugam, Ronny Luss
New Oracle-Efficient Algorithms for Private Synthetic Data Release
Giuseppe Vietri, Steven Wu, Mark Bun, Thomas Steinke, Grace Tian
Safe Reinforcement Learning in Constrained Markov Decision Processes
Akifumi Wachi, Yanan Sui
Min-Max Optimization without Gradients: Convergence and Applications to Black-Box Evasion and Poisoning Attacks
Sijia Liu, Songtao Lu, XIANGYI CHEN, Yao Feng, Kaidi Xu, Abdullah Al-Dujaili, Mingyi Hong, Una-May O’Reilly
Improving the Sample and Communication Complexity for Decentralized Non-Convex Optimization: Joint Gradient Estimation and Tracking
Haoran Sun, Songtao Lu, Mingyi Hong
Proper Network Interpretability Helps Adversarial Robustness in Classification
Akhilan Boopathy, Sijia Liu, Gaoyuan Zhan), Cynthia Liu, Pin-Yu Chen, Shiyu Chang, Luca Daniel
Stochastic Gauss-Newton Algorithms for Nonconvex Compositional Optimization
Quoc Tran-Dinh, Nhan H Pham, Lam Nguyen
Learning to Rank Learning Curves
Martin Wistuba, Tejaswini Pedapati
Model Fusion with Kullback–Leibler Divergence
Sebastian Claici, Mikhail Yurochkin, Soumya Ghosh, Justin Solomon
Invariant Risk Minimization Games
Kartik Ahuja, Karthikeyan Shanmugam, Kush Varshney, Amit Dhurandhar
Is There a Trade-Off Between Fairness and Accuracy? A Perspective Using Mismatched Hypothesis Testing
Sanghamitra Dutta, Dennis Wei, Hazar Yueksel, Pin-Yu Chen, Sijia Liu, Kush Varshney
Learning Task-Agnostic Embedding of Multiple Black-Box Experts for Multi-Task Model Fusion
Nghia Hoang, Thanh Lam, Bryan Kian Hsiang, Patrick Jaillet
Transfer Learning without Knowing: Reprogramming Black-box Machine Learning Models with Scarce Data and Limited Resources
Yun Yun Tsai, Pin-Yu Chen, Tsung-Yi Ho
Shiyu Chang, Yang Zhang, Mo Yu, Tommi Jaakkola
Unsupervised Speech Decomposition via Triple Information Bottleneck
Kaizhi Qian,Yang Zhang, Shiyu Chang, Mark Hasegawa-Johnson, David Cox
Curvature-corrected learning dynamics in deep neural networks
Ben Dongsung Huh
Fast Learning of Graph Neural Networks with Guaranteed Generalizability: One-hidden-layer Case
Shuai Zhang, Meng Wang, Sijia Liu, Pin-Yu Chen, Jinjun Xiong
Bio-Inspired Hashing for Unsupervised Similarity Search
Chaitanya Ryali, John Hopfield, Leopold Grinberg, Dmitry Krotov
Srinivasan Arunachalam, Reevu Maity
Two Simple Ways to Learn Individual Fairness Metric from Data
Debarghya Mukherjee, Mikhail Yurochkin, Moulinath Banerjee, Yuekai Sun
PoWER-BERT: Accelerating BERT Inference via Progressive Word-vector Elimination
Saurabh Goyal, Anamitra Roy Choudhury, Venkatesan Chakaravarthy, Saurabh Raje, Yogish Sabharwal, Ashish Verma
Not Your Grandfather’s Test Set: Reducing Labeling Effort for Testing
ICML Challenges in Deploying and Monitoring Machine Learning Systems
Begum Taskazan, Jiri Navratil, Matthew Arnold, Anupama Narasimha Murthi, Ganesh Venkataraman, Benjamin Elder
PaccMann^RL on SARS-CoV-2: Designing antiviral candidates with conditional generative models
ICML workshop for Computational Biology
Jannis Born, Matteo Manica, Joris Cadow, Greta Markert, Nil Adell Mill, Modestas Filipavicius, Maria Rodriguez Martinez
Leveraging Simple Model Predictions for Enhancing its PerformanceICML Workshop on XXAI: Extending Explainable AI Beyond Deep Models and Classifiers
Amit Dhurandhar, Karthikeyan Shanmugam, and Ronny Luss
On the Equivalence of Bi-Level Optimization and Game-Theoretic Formulations of Invariant Risk Minimization
ICML Workshop on Inductive Biases, Invariances, and Generalization
Kartik Ahuja, Karthikeyan Shanmugam, Kush R. Varshney, Amit Dhurandhar
Chi-square Information for Invariant Learning
ICML Workshop on Uncertainty and Robustness in Deep Learning
Prasanna Sattigeri, Soumya Ghosh, Samuel Hoffman
Prediction of neonatal mortality in Sub-Saharan African countries using data-level linkage of multiple surveys
ICML 2020 ML for Global Health
Girmaw Abebe Tadesse, Celia Cintas, Skyler Speakman, Komminist Weldemariam
Machine Learning to Assess the Association of H. Pylori Infection and Gastric and Oesophageal Cancer by Detecting Western Blot Protein Bands
ICML 2020 ML for Global Health
Girmaw Abebe Tadesse, Daniel Chapman, Ling Yang, Pang Yao, Iona Millwood, Zhengming Chen, Tingting Zhu, Omid Aramoon, Pin-Yu Chen, Gang Qu
Consumer-Driven Explanations for Machine Learning Decisions: An Empirical Study of Robustness
ICML Workshop on Human Interpretability in Machine Learning (WHI)
Michael Hind, Dennis Wei, Yunfeng Zhang
Do You Sign Your Model?
Workshop on Challenges in Deploying and Monitoring Machine Learning Systems
Omid Aramoon, Pin-Yu Chen, Gang Qu
Hierarchically Attentive Graph Pooling with Subgraph Attention
ICML 2020 Workshop on Graph Representation Learning and Beyond (GRL+ 2020)
Sambaran Bandyopadhyay, Manasvi Aggarwal, M. Narasimha Murty
Unsupervised Attention-Guided Atom-Mapping
ICML 2020 workshop for ML Interpretability for Scientific Discovery
Philippe Schwaller , Benjamin Hoover, Jean-Louis Reymond, Hendrik Strobelt, Teodoro Laino
Explaining Chemical Toxicity Using Missing Features
ICML 2020 workshop for ML Interpretability for Scientific Discovery
Kar Wai Lim, Bhanushee Sharma, Payel Das, Enara Vijil, John Dordick
Efficient Black-Box Combinatorial Optimization
Workshop on Real World Experiment Design and Active Learning at ICML 2020
Hamid Dadkhahi, Karthikeyan Shanmugam, Jesus Rios, Payel Das
Keynotes at Workshops
International Workshop on Federated Learning for User Privacy and Data Confidentiality in Conjunction with ICML 2020 (FL-ICML’20)
LatinX in AI (LXAI) Workshop
ICML 2020 Workshop on Human Interpretability in Machine Learning (WHI)
Adrian Weller, Alice Xiang, Amit Dhurandhar, Been Kim, Dennis Wei, Kush Varshney, Umang Bhatt