New Thinking

Three Ways Metalearning Will Change the Face of Artificial Intelligence

Share this post:

If you were going purely off of what is depicted in movies and television shows, you’d think that using AI was an extremely streamlined process. Got a problem? Just push a few buttons or simply say what you want to your computer—and a few moments later the AI will have the issue resolved.

The reality is that the everyday work of machine learning can be an extremely manual, painstaking process. “AI” isn’t just a single tool like a wrench or a power drill. Instead, it’s a whole set of methods and approaches that need to be fit together and blended to solve the particular challenge at hand. That still takes a fair amount of human judgment, and hard-won intuition.

As it stands today, you need a few major components to successfully get an AI system to do what you want it to do. You need data—a large set of examples that help teach the machine a task. You need compute—the processing power to actually do the training. And finally, you need a good learning architecture—in effect, a process by which the machine will learn.

This last piece—designing an effective learning architecture—can be a time-consuming and expensive step in the design of an AI system. It requires specialized expertise, and frequently proceeds by trial-and-error. The problem is likely to get worse going forward, too. One general finding in the recent wave of breakthroughs in machine learning is that more complex architectures frequently are better at accomplishing a task. But as the level of complexity increases, so too will it be more challenging for human specialists to keep up.

In light of this, researchers have been focused on an intriguing and somewhat head-spinning possibility: what if designing this architecture could be thought of as itself a machine learning problem? That is, rather than having humans tailoring “artisanal” AI systems by hand, what if AI systems were simply designed by other AI systems?

This idea, referred to as metalearning in the field, has been seeing some remarkable successes in recent years. For one, these “learning to learn” approaches have been able to generate architectures which outperform hand-designed competitors. Interestingly, there is also some evidence that these methods are sometimes able to uncover new, better designs that haven’t yet occurred to human specialists.

There have been other outcomes, as well. Metalearning techniques appear to be opening the way for AI systems that are increasingly able to accomplish a whole range of tasks, rather than the specific one they were trained to solve. Moreover, these approaches can grant AI systems the benefit of knowledge accumulated in earlier tasks, lowering the amount of data needed to teach the machine on the new task.

This promising progress is driving a wave of work in the space, with an active community of researchers pushing these methods ahead. As it develops, metalearning is poised to have three major impacts on the business landscape around AI.

First and most obviously, metalearning can significantly speed up the design of effective AI systems. Metalearning approaches can produce learning architectures that perform better than their hand-crafted counterparts, and might do so faster than a team of human specialists. This is relevant as policymakers and the industry attempt to predict the likely impact of AI breakthroughs on the economy at large. Accelerating the creation of effective AI systems might increase the rate at which it is implemented in different sectors. It might also encourage the technology to be deployed in sectors where it was previously unprofitable to do so, since it lowers the cost to creating AI systems that can take on a given task. In either case, a fast pace of progress in metalearning might portend an increase in both the rate, scale, and scope of automation throughout society.

Second, machine learning expertise may become increasingly commoditized. At the moment, the need for AI systems to be hand-crafted means that the demand for machine learning specialists has been very intense. Metalearning could alter this balance. The task of designing basic learning architectures seems slated for increasing automation, and metalearning may automate the design of more complex systems as well. This might decrease the need to maintain large, expensive teams of engineers and researchers to design and deploy AI systems. At the same time, metalearning might also significantly expand the types of businesses and industries trying to implement these technologies as it lowers the cost to using AI. Talent in the space might therefore become cheaper and more distributed over time.

Third, metalearning may significantly raise the importance of having access to computational power. Metalearning arguably lowers the need for data to create effective AI systems in two ways. It assists in the development of more general systems, which can transfer knowledge from one context to another. This reduces the amount of data you need in solving problems in the new context. It also potentially helps to create more effective learning architectures, which can reach higher proficiency with less training data. But, metalearning is also computationally intensive in that it requires lots of processors to pull off in a reasonable length of time.

This changes the competitive landscape around AI systems. Access to the right data and sufficient data has often been the key factor in getting AI systems to work well. As a result, companies possessing massive datasets in a specific domain have had a significant advantage in benefitting from the recent breakthroughs in machine learning.

Metalearning might offset this advantage in the future by lowering the data needs necessary to compete. Companies with data only somewhat related to a particular problem, or less data than an industry leader, may benefit from metalearning techniques that allow the creation of high-quality AI systems on par with competitors with a data advantage. Simultaneously though, it will raise the importance of access to a new key resource: computational power.

Metalearning is still in its early stages, and the research still must make the jump to broader commercial use. That being said, it’s well worth keeping an eye on. Increasingly, the business opportunities and risks in AI will not depend just on whether or not machine learning can accomplish a certain task. How it does so will become an equally important if not greater consideration.

Learn more about IBM iX

More New Thinking stories

Dreamforce ’18: Paige O’Neill, CMO, Sitecore

Amanda Thurston (Editor-in-Chief, thinkLeaders) sits down with Bluewolf Women Innovators Network (WIN) Honoree Paige O’Neill (CMO, Sitecore) to discuss the state of women leaders in the workplace and how organizations and leaders can drive change to create an inclusive work environment. Follow @IBMthinkLeaders on Twitter > Join the IBM thinkLeaders LinkedIn group > Subscribe to […]

Continue reading

Engaging for Impact: How Women Executives Are Changing the Workplace

Female leaders drive higher levels of employee engagement and satisfaction within the workplace. IBM thinkLeaders and the Women Innovators Network present this Dreamforce 2018 panel with three executive women on how they are impacting change in their institutions, bridging generation gaps, and fostering the future pipeline of diverse talent. Panelists: (Moderator) Debbie Vavangas – DS&iX […]

Continue reading

Dagne Dover Founders Panel

IBM iX thinkLeaders presents a fireside chat with the founders of Dagne Dover: – Melissa Mash, CEO – Jessie Dover, Creative Director – Deepa Gandhi, COO

Continue reading