Posted in: IBM Research-Zurich, Nanotechnology, Storage

Unsupervised learning with artificial neurons

Manuel Le Gallo’s research will inspire a new generation of extremely dense neuromorphic computing systems)

Manuel Le Gallo’s research will inspire a new generation of extremely dense neuromorphic computing systems. (Source: IBM Research – Zurich)

Inspired by the way the human brain functions, a team of scientists at IBM Research in Zurich, have imitated the way neurons spike, for example when we touch a hot plate. These so-called artificial neurons can be used to detect patterns and discover correlations in Big Data with power budgets and at densities comparable to those seen in biology, something which scientists strived to accomplish for decades. They can also learn, unsupervised at high speeds using very little energy.

The paper entitled “Stochastic phase-change neurons,” which appeared today on the cover of Nature Nanotechnology, outlines the research and its findings.

I spoke with the paper’s co-author and IBM Research – Zurich scientist Manuel Le Gallo, who is currently working on a PhD from ETH Zurich.

How does an artificial neuron function?

Manuel Le Gallo: The neuron has a certain functionality we call “integrate and fire.” The neuron acts as an accumulator — if you keep sending multiple inputs to the neuron, the neuron will integrate all those inputs. Depending on the amount of inputs and their strengths, the membrane potential will reach a certain threshold, and the neuron will “fire” or “spike.” Such an accumulator can be used to perform surprisingly complex computational tasks.

How does the human brain inspire the development of artificial neurons?

ML: The artificial neuron is built to mimic what a biological neuron does. An artificial neuron won’t have the exact same functionality but is still close enough that you can achieve the computation performed by the brain using these neurons. Usually, artificial neurons are built using CMOS-based circuits, the standard transistor technology we have in our computers. Our paper focuses on using non-CMOS devices, such as phase-change devices, to reproduce similar functionality at reduced power consumption and increased areal density.

What is your contribution to the paper?

ML: We obtained an understanding of phase-change device physics from the characterization and modeling work I have been doing in the past three years. This was crucial in order to design artificial neurons based on these devices and to understand their functionality. In addition, I obtained some of the experimental data presented in the paper and contributed to the analysis and interpretation of the results.

“We think our approach will be more efficient, especially for processing large amounts of data.”

—Manuel Le Gallo, IBM Research scientist

In which contexts can artificial neurons be applied?

The tiny squares are contact pads that are used to access the nanometer-scale phase-change cells (not visible). The sharp probes touch the contact pads to change the phase configuration stored in the cells in response to the neuronal input. Each set of probes can access a population of 100 cells.

The tiny squares are contact pads that are used to access the nanometer-scale phase-change cells (not visible). The sharp probes touch the contact pads to change the phase configuration stored in the cells in response to the neuronal input. Each set of probes can access a population of 100 cells.

ML: In our paper, we demonstrate how you can detect correlations from multiple streams of events. Assume that you have multiple streams of binary events and you want to find which streams are temporarily correlated, for example, when the 1s occur concurrently.

In the paper, we demonstrated how you can detect correlations from multiple streams of events.

What do you mean by events?

ML: Events could be, for example, Twitter data, weather data or sensory data collected by the Internet of Things. Assume that you have multiple streams of binary events and you want to find which streams are temporarily correlated, for example when the 1s come concurrently. We show in the paper how we could do this discrimination using just one neuron connected to multiple plastic synapses receiving the events.

What makes neuromorphic computing more efficient than conventional computing?

ML: In conventional computing, we have a separate memory and logic unit. Whenever you want to perform a computation, you must first access the memory, obtain your data, and transfer it to the logic unit, which returns the computation. And whenever you get a result, you have to send it back to the memory. This process goes back and forth continuously. So if you’re dealing with huge amounts of data, it will become a real problem.

In a neural network, computing and storage is co-located. You don’t have to establish communication between logic and memory; you just have to make appropriate connections between the different neurons. That’s the main reason we think our approach will be more efficient, especially for processing large amounts of data.


Manuel Le Gallo came to Zurich to pursue his Master’s degree in Electrical Engineering at the Swiss Federal Institute of Technology (ETH Zurich). He completed his thesis at IBM, where a job offer fit his background and interests. He is currently working on his PhD.

About the author: Millian Gehrer is a summer intern at IBM Research – Zurich, where he is interviewing scientists to learn more about their work and motivations. In the fall, he will begin studying Computer Science as an undergraduate at Princeton University. 

Comments

  1. Jona says:

    There’s certainly a lot to know about this topic.
    I like all the points you made.

  2. skierpage says:

    Could this be used to make existing brain simulators (including IBM’s rat cortical column, cat brain. whole brain, and others I’ve forgotten) better-faster-cheaper?

  3. Cynthia Pulver says:

    The magic of stepping stones…keeping stepping forward.

  4. Paul says:

    Thanks for the detailed post.

Add Comment

Your email address will not be published. Required fields are marked *

Millian Gehrer

Millian Gehrer

Intern, IBM Research - Zurich