Seeing and feeling shapes described by a stream of sound

Share this post:

Researchers at IBM’s lab in Tokyo have developed a new method to convert visual shapes on the web into a real-time stream of sound to help visually impaired better understand the shape of an object shown on a web site.

Still an early prototype, the technology converts the shape of an object, such as an image shown on a webpage, into a stream of sound. By listening to the sounds, the visually impaired can mentally feel the shape or better-understand turn-by-turn directions. The technology may also be useful as a real-time guide for online maps which shows route to destination via a mobile phone.

According to a preliminary experiment conducted by the IBM researchers, both blind and sighted people quickly adapted to following directional sound, and recognizing shapes such as randomly drawn paths, letters in the alphabet, and numbers. People with musical training tended to adopt the technology at a faster rate. And some of the participants felt as if they could mentally “see” and “feel” the shapes described by the sounds.

The prototype has the potential to not only help the visually impaired, but to also benefit any user of a mobile device. In situations where a person could use the assistance of a mobile device, but cannot easily look at a screen; or in instances when not all visual information is present (such as in online gaming), the system can provide real-time guidance in the real world or a virtual world.

IBM researchers in Tokyo are advancing the prototype technology with the goal of helping empower the visually impaired to better-navigate the ever increasing non-textual, visual information on the web.

More stories

Real-Time Sequential Decision-Making by Autonomous Agents

A new approach to real-time sequential decision-making represents a step towards autonomous agents that can make critical decisions in real time.

Continue reading

How inventing became a passion: Developing autonomous vehicles to help support healthy aging and people with disabilities

For more than 10 years, I have focused my research on creating accessible technology for people with disabilities. This led to my current research in designing solutions for the aging population whose cognitive and functional ability change over time. Over the weekend I had the opportunity to participate in the USPTO’s press event at SXSW, […]

Continue reading

Emerging Leaders: Female scientists driving our global research agenda

As March 8, 2018 marks International Women’s Day, this year’s campaign is a #PressforProgress – focusing on gender parity in the community and in the workplace. Since early days at IBM, we have always been led by Thomas J. Watson Jr.’s famous 1953 memo: “It is the policy of this organization to hire people who […]

Continue reading