The breakthrough that has not yet happened that will completely rearrange the current landscape of AI is using an extremely small dataset to train AI systems. Right now, AI requires very large training data sets to learn. And we have proof in the human toddler that we can actually have learning with very small data sets. Somebody in the future will figure out how to do that well. That will be a really huge shift, and it will be very liberating in many ways.
I think another one in the future is unsupervised learning, where the machine learns largely on its own. We're only just beginning to deal with that.
Besides those, there’s a real need for symbolic reasoning and alternative routes to intelligence that I believe are going to be necessary to make more robust AI tools. We've been exploring how far you can go with deep learning and some people think we can go all the way. That may be so, but I think deep learning is just one mechanism that needs to be used with others, much in the way that a gear or a pendulum is just one mechanism that makes a clock work.