AI’s Interface Revolution

By | 5 minute read | August 20, 2019

Here are a few actions that you probably perform hundreds of times a day at your computer without giving it a second thought: clicking and dragging a file, dropping a file into a folder, closing and opening windows. These actions are so routine and commonplace that you’d likely be confused if you sat down at a computer and found that you weren’t able to do these things.

But things weren’t always this way. For decades, computers lacked any interface that was usable by non-programmers. Computers were text-based, requiring a deep knowledge of arcane commands and syntax. Older computers lacked even that, relying on physical sets of paper cards with holes punched in them to execute programs. Achieving a simple task like “dropping” a file into a “folder” required much more than the simple point-and-click action that you’ve done countless times on your computer and phone.

Thanks to the work of researchers like Douglas Englebart, Alan Kay, and Ivan Sutherland throughout the 1960s, ‘70s, and ‘80s, people began to be able to experience computers visually. A file could be represented by an icon of a sheet of paper. A mouse could be used to move that file around on a screen. That file could be “picked up” and placed into a file directory, which would be represented by an icon of a folder. This way of doing things — called a “graphical user interface” or GUI — fundamentally changed the way that we use computers.

This history is important because it teaches us about what happens to a technology when interfaces are streamlined and become usable by non-specialists. Such a lesson is applicable to our current position on the cusp of the AI and machine learning revolution.

Despite the excitement for the scientific breakthroughs in AI, machine learning has long been a technology for trained specialists. As an non-technical consumer or a small business, it has been difficult to get a machine learning system “off the shelf” and immediately start applying it to a problem. Training and building custom machine learning models to take on new tasks has often required hiring from a small pool of experienced researchers with PhDs.

This has been, in part, an interface problem. Similar to computers in the 1960s, machine learning has, for much of its history, required deep knowledge and technical expertise. As a result, its use has been limited to a narrow set of tasks and a narrow set of businesses with sufficient resources to acquire the right talent.

This may be changing. The last few years have seen the launch of a number of new products that aim to make machine learning easy to use for a much larger audience. This has been done through the development of GUIs that work to empower non-coders to analyze, train, and deploy machine learning systems on their own. An example of this is IBM’s Watson Studio.

Obviously, a GUI is not just one thing — the future user interfaces for machine learning could make it easier to handle many different parts of the workflow around the technology. As in the early days of personal computing, the market is still sorting out what kinds of interfaces will be most useful for specialists and non-specialists alike. However, the state of play in the research field and the trends in these recent products suggest that creating visual metaphors in the following places might have the biggest impact:

  • Model Design: What if you could design and revise machine learning models visually? Many products are starting to play with visual editors that represent machine learning architectures as 2D flowcharts. To manage ever more complex model structures, we might eventually land on 3D visualization platforms, very much the same way that products like AutoCAD support architects and mechanical engineers. Such a tool could speed the time-consuming process of building and retooling machine learning models.
  • Interactive Training: Even if non-specialists aren’t in the business of building learning architectures from scratch, they will increasingly need a quick way to visualize the improvement of a machine learning system as it trains on their data over time. Future platforms might help to visually represent cases of high and low-confidence predictions made by a model to aid in assessing performance over multiple epochs of training.
  • Interrogating Data: Data is complex, and it is not always clear where biases might be lying in wait to generate flawed machine learning systems that act in ways their designers did not intend. Specialists and non-specialists both will need simple, powerful visual tools that make it easier to sift through multi-dimensional data and identify places where better or relevant data should be collected, even before training occurs.
  • Debugging and Monitoring: You might not need to be able to fix a complex machine learning system all on your own, but it is important to know when an AI system might be misbehaving in ways that require a check-up.  These types of warning indicators will become increasingly important for signaling to non-experts when the technology may be unreliable or unsafe.

The arrival of GUIs in computing offer some hints of what the implications for the arrival of GUIs in the machine learning market might have. For one, GUIs massively expanded the range of people using computers, and in doing so also fostered a market for a much larger range of uses for computers, from games and music players to image editing and social media.

The same might be the case in the AI space. As the tools get easier to use by a larger audience, we may also discover that machine learning has applications in a range of domains that we might not have originally anticipated. AI is already being used by a growing community of DIY tinkerers who are applying it to everything from sorting laundry to building autonomous model cars. We will see these applications grow in diversity and scope over time.

It took decades for the first GUIs to evolve into the familiar interfaces that we rely on today in our computers. We’re at the beginning of that process for AI. As more businesses seek to integrate the technology into their products and services, interface design is poised to be a major area of investment, innovation, and competition in the coming years.