Cognitive Computing

Thinking design in the lab

Share this post:

If you stand outside Almaden’s auditorium long enough, you will hear a series of gentle pings to your right. Look that way and you’ll see four white marbles, each bouncing around the insides of its own glass terrarium. These bulbous terrariums, small enough to cradle in the palm of your hand, hang from black bungee cords, in turn attached to pipes. Every so often, a flashing sensor mounted on a pipe will send signals to a tiny motor right next to it, spurring it to nudge a bungee cord. The movement of the cord triggers the marbles to popcorn-pop around the terrarium – ping, ping, ping, ping.


Raphael Arar sits next to his Wayfinding installation.

Researcher and designer Raphael Arar recently featured this Wayfinding art installation at IBM Research – Almaden’s 30th anniversary event. He sought to demonstrate data visualization by combining design-thinking with the scientific themes of big data analytics. Wayfinding uses proximity data collected from four sensors as cues to create sounds using the marbles in the terrarium.

Arar is one of the first designers at IBM’s Almaden research lab collaborating with researchers to fuse art and engineering. He explores user experience and user interaction and Internet of Things as a principal designer, as well as data visualization art through projects like Wayfinding.

“My artwork is a balance between fulfilling personal creative desires, and asking questions that I want to learn more about,” said Arar. “I want to explore the medium of technology as more of an aesthetic and I use a design process to get me there.”

Arar’s dual interests in art and engineering are rooted in his childhood. As the son of a civil engineer and a high school math teacher in Ohio, Arar developed a strong foundation in the sciences. Yet he also felt compelled to explore his passion for art and music. After spending years pursuing engineering and the arts in separate educational tracks, Arar came to a revelation.

“I realized the two didn’t have to be so different,” said Arar. “Art can inform science, and science can inform art.”

He began to explore the design and technology field, and as he did so, the mediums he worked with began to transform.

“Drawing and painting transformed to generative software art,” said Arar, describing how his artistic pursuits changed over time. “Music and sound became increasingly driven by handmade circuits, and the sculptural work became more and more electromechanical.”

Arar laughs while wearing 3-D glasses.

Arar trying out 3-D glasses.

After earning his MFA from Cal Arts, Arar worked with advertising agencies and start-up companies on their product designs, before joining IBM in 2014 as the design lead for the IBM-Apple partnership. From there, he moved to Research, and is now a designer and researcher at IBM Research-Almaden, where explores the perfect intersection of his interests.

“As an artist, I seek to ask questions to inspire others to be curious about the way we operate as humans – both with ourselves and with others,” said Arar.

“And as a designer, I seek to answer questions to aid others in better navigating the world around them. I use my aesthetic process as a way to cultivate out-of-the-box thinking. And I approach this exploration as a designer, leveraging aspects of processes rooted in engineering principles, as well as more contemporary design thinking approaches to problem-solving.”

And according to Arar, IBM Research serves as the ideal place to combine art and design engineering, thanks to its computing innovation legacy and its focus on complex problem-solving.

“With its current focus on cognitive computing, big data, and design, the company’s very essence explores themes that I’m naturally interested in,” said Arar. “How do we navigate the increasingly complex situations we face as individuals and groups? How can technology empower us to be our best selves, and how can it be used to facilitate collaboration? These are questions I’m curious about, and I’m fortunate to be able to explore them both with the projects I’m working on.”

Finding your way

One of these projects is Wayfinding, which Arar uses to explore how user interaction and data visualization can affect one another.

“Many of us are familiar with the phrase, ‘find your way,’” said Arar, explaining his thought process behind the installation. “The piece seeks to reflect on the paths one chooses in life and those paths’ subsequent periods of chaos and harmony.”

Each of the four sound sculptures that compose Wayfinding represents a cardinal direction – North, South, East, and West – emphasizing the art installation’s theme of navigation. As users move closer to the installation, Wayfinding will respond to changes in proximity; the sensors on each sound sculpture will trigger the motors to nudge the bungee cord according to detected movement. Wayfinding collects proximity data from these sensors, and this data is visualized on a dashboard display next to the installation.

The data collected by the sensors on Wayfinding is visualized in a digital dashboard.

The data collected by the sensors on Wayfinding is visualized in a digital dashboard.

“The piece forms a question-and-answer model with data generation and data visualization,” said Arar. “The physical component invites people to interact and generate data, while the digital component serves as a way to visualize it.”

In this way, Arar says, the installation and its users form a feedback loop of data, creating a unique user experience, while also harnessing data to create art. This new approach to data promotes creativity and exploration in the research lab.

“As designers, researchers, and technologists, we spend a lot of time trying to answer questions the business and our clients need using existing data, analytics, and sensors,” said Arar. “We rarely take the time to explore unique outlets of the technology at hand – which could lead to questions that help us think differently. At IBM, we pride ourselves in innovative thinking; this piece and its underlying themes help get us there.”

In addition to Wayfinding, Arar is constantly brainstorming and working on other projects that allow him to examine technology and the research field through different ways of thinking.

“I want my work to push people to the edge of tech, and to think about how design impacts our daily lives, inside and outside of the workplace,” said Arar. “Being in research allows me to combine art, science, and design-thinking and operate at that intersection to the best of my ability.”

More about Raphael’s work: Raphael’s artwork has been shown at museums, conferences, festivals and galleries internationally including the Espoo Museum of Modern Art, International Symposium on Electronic Art (ISEA), Gamble House Museum, Boston Cyberarts Gallery, and Athens Video Art Festival. Recently, he submitted three projects to the 2017 SXSW Conference: Applying Science to Conversational UX Design, Cognitive UX, and Recursive Computational Art: DataGen meets DataViz.

About the author: Kelly Shi is a Communications intern at IBM Research-Almaden, where she is excited to produce media content about IBM researchers and their teams. In the fall, she will return to Northwestern University and will continue working towards her BA in Communication Studies.

More Cognitive Computing stories

Unlocking the Potential of Today’s Noisy Quantum Computers for OLED Applications

Scientists at Mitsubishi Chemical, a member of the IBM Quantum Hub at Keio University in Japan, reached out to our team about experimenting with new approaches to error mitigation and novel quantum algorithms to address these very challenges. In the new arXiv preprint, “Applications of Quantum Computing for Investigations of Electronic Transitions in Phenylsulfonyl-carbazole TADF Emitters,” we – along with collaborators at Keio University and JSR - describe quantum computations of the “excited states,” or high energy states, of industrial chemical compounds that could potentially be used in the fabrication of efficient organic light emitting diode (OLED) devices.

Continue reading

New Macromolecule Could Hold Key to Reversing Antibiotic Resistance

To address the challenge of antibiotic resistance, scientists from IBM and the Agency for Science, Technology and Research and the Singapore-MIT Alliance for Research and Technology have published new findings in Advanced Science, which unveil the effectiveness of a new polymer in the fight against resistant bacteria.

Continue reading

IBM Research at SIGMOD 2020

ACM SIGMOD/PODS 2020 like many other events impacted by COVID-19 pandemic will be taking place virtually from June 14 through June 19. The focus of work at SIGMOD 2020 ranges from adding graph querying to relational databases, to natural language interfaces to data, to operationalizing data for new AI workloads. Results to be presented includes work done at our IBM Research-Almaden and IBM Research-India labs, as well as by our summer interns from universities and our partners in other IBM units.

Continue reading