AI

Exploring Frontiers of AI at ICLR 2018

Share this post:

IBM Research is a pioneer across many aspects of AI. At the 6th International Conference on Learning Representations (ICLR 2018), our team will share recent discoveries in learning data representations, techniques that are key to the success of machine learning algorithms. These techniques enable machine learning systems to automatically discover how to represent raw data for subsequent analysis. Learning data representations is an important learning task that powers computer vision, speech recognition, natural language processing, drug design, and other advances in AI. At ICLR 2018, IBM Research will present technical papers on adversarial learning, self-organizing networks for multi-task learning, open-domain question answering, disentanglement of representations, reinforcement learning, and deep learning for graphical data. Below are details on the papers IBM Research will be presenting at ICLR 2018.

In one paper, IBM Research will be introducing the concept of a “routing network,” a kind of self-organizing neural network consisting of two components: a router and a set of function blocks. Given an input, the router makes a routing decision, choosing a function block to apply and passing the output back to the router recursively. This allows similar tasks to share layers, while enabling differing tasks to choose distinct layers. Routing networks perform much better than alternative approaches on a set of multi-task learning benchmarks.

Another IBM AI project featured at ICLR 2018 involves a new metric called CLEVER that can be used to evaluate the robustness of a neural network against attack. The CLEVER score calculates the minimum strength that an adversarial attack would need in order to succeed at deceiving the neural network. Higher scores indicate more robust neural networks. The scores are attack-agnostic, meaning they can be applied to existing and unseen attacks, and are computationally feasible for large neural networks. CLEVER scores can be used to compare the robustness of different network designs and training procedures to help researchers build more reliable AI systems.

IBM Research is a gold sponsor of ICLR 18. We will be at the conference expo at booth #500, where you can try your hand at attacking imaginary banks’ AI check image processing systems by distorting check digits and using CLEVER scores to guide your decisions and maximize your profits.
Big Check demo

Accepted papers at ICLR 2018

Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach
Tsui-Wei Weng, Huan Zhang, Pin-Yu Chen, Jinfeng Yi, Dong Su, Yupeng Gao, Cho-Jui Hsieh, Luca Daniel

Sobolev GAN
Youssef Mroueh, Chun-Liang Li, Tom Sercu, Anant Raj, Yu Cheng

Routing Networks: Adaptive Selection of Non-linear Functions for Multi-Task Learning
Clemens Rosenbaum, Tim Klinger, Matthew Riemer

Evidence Aggregation for Answer Re-Ranking in Open-Domain Question Answering
Shuohang Wang, Mo Yu, Jing Jiang, Wei Zhang, Xiaoxiao Guo, Shiyu Chang, Zhiguo Wang, Tim Klinger, Gerald Tesauro, Murray Campbell

Variational Inference of Disentangled Latent Concepts from Unlabeled Observations
Abhishek Kumar, Prasanna Sattigeri, Avinash Balakrishnan

Eigenoption Discovery through the Deep Successor Representation
Marlos C. Machado, Clemens Rosenbaum, Xiaoxiao Guo, Miao Liu, Gerald Tesauro, Murray Campbell

FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling
Jie Chen, Tengfei Ma, Cao Xiao

More AI stories

A Highly Efficient Distributed Deep Learning System For Automatic Speech Recognition

In a recently published paper in this year’s INTERSPEECH, we were able to achieve additional improvement on the efficiency of Asynchronous Decentralized Parallel Stochastic Gradient Descent, reducing the training time from 11.5 hours to 5.2 hours using 64 NVIDIA V100 GPUs.

Continue reading

IBM Research advances in end-to-end speech recognition at INTERSPEECH 2019

IBM scientists presented three papers at INTERSPEECH 2019 that address the shortcomings of End-to-end automatic approaches for speech recognition - an emerging paradigm in the field of neural network-based speech recognition that offers multiple benefits.

Continue reading

High quality, lightweight and adaptable Text-to-Speech (TTS) using LPCNet

Recent advances in deep learning are dramatically improving the development of Text-to-Speech systems through more effective and efficient learning of voice and speaking styles of speakers and more natural generation of high-quality output speech.

Continue reading