Artificial Intelligence

How an app designed for people who are blind could solve a global skills shortage

Share this post:

When Karen Kilroy volunteered to coach the dragon boat racing team at the Texas School for the Blind and Visually Impaired (TSBVI), she didn’t expect to come out of it with an idea for a new application for AI—and a way to help solve a growing problem in the global talent pool.

But that’s what happened one day last year when the Austin-based CEO of Kilroy Blockchain was leading a group of students in practice for the team sport, which requires crews of 20 paddlers to propel 40-foot-long boats on open water.

In between bouts on the water, Kilroy noticed that the students were spending a lot of down time on their phones. What if, she wondered, an app could allow them to use those devices to explore the world beyond their screens?

“This seemed really exciting to me, because the right application would have the potential to open up a new world of interaction and information for the visually impaired. Back at the office, my team and I got to work on refining our solution idea,” she told IBM.

Soon after, Kilroy learned about IBM’s inaugural Watson Build Challenge, a global initiative designed to spark the development of AI-based solutions. It was just the platform she needed to help turn her idea into a reality.

Along with more than a thousand other IBM business partners, Kilroy pitched her idea, an app called “Riley (RealLife Adventure),” to the global competition, and was ultimately selected as the winner for North America.

The app, which officially launched at the Consumer Electronics Show in January, has two main features. One, “What’s That?” uses Watson Visual Recognition to identify objects in a user’s photo and audibly describes it through speech-to-text technology. The other, “Look Around,” uses Watson Content Hub to provide an audible description of a user’s surroundings based on their geolocation.

“We often train [visually impaired] students to use telescopes and binoculars to look at what’s around them, but it makes them kind of different and teenagers don’t want to look different. So being able to explore their surroundings using a program like Riley has a positive psychological effect, particularly on teenagers,” said TSBVI Superintendent Bill Daugherty.

This fall, select students at TSBVI will test that theory when they take Riley for a spin as part of a pilot program. Primarily, they’ll be looking to evaluate how the app can be used to help overcome sight-related barriers.

But in the future, said Kilroy, the app could also be used to tackle a major barrier facing business across industries—particularly, a growing skills shortage. According to a recent report by the Bpifrance public investment bank, the Financial Times reports, nine out of 10 mid-sized companies are facing recruitment difficulties. The main recruitment issue, the companies report, is a lack of qualified workers.

Riley, said Kilroy, could help close that gap.

According to Kilroy, Riley’s server can be configured to become a visual subject matter expert for any business’ products. It could, for instance, be trained to recognize and distinguish specific types of produce, equipment in a coffee bar, or parts on an assembly line. It could then be used to train workers on how best to interact with those products.

“It’s actually less of a heavy lift, because Riley doesn’t have to be able to describe the whole universe— just a narrow slice of it,” said Amanda Lacy, a quality assurance technician for Riley who has been blind since birth.

People who are blind and visually impaired are already using Riley to explore their world. Now, Kilroy said, the next big “real life adventure” awaits companies looking to take enable their workforces with a game-changing training tool.

“Maybe the limitation is sight, or maybe it’s language. Maybe the limitation is knowledge. What we’re looking for are companies that want to try out ‘new collar’ workforce tools by supplying their workforce with tools that enable people to do jobs they weren’t able to do in the past,” Kilroy said.

For more stories of industry transformation, download Industrious magazine.

Content Producer, IBM Industries

More Artificial Intelligence stories

With AI-powered translation, industries look to speak every customer’s language

Written by Jordan Teicher and Joseph Rakowski | November 20, 2018 | Artificial Intelligence, New Normal, Retail and Consumer Products...

In customer service, Lingmo's Danny May says, communication is crucial. That's why companies are turning to AI-powered translation tools. ...read more


In one of the coldest places on Earth, this utility keeps the power on

Written by Rich McKay | November 9, 2018 | Artificial Intelligence, Data, Energy and Utilities...

How does NB Power keep homes warm and its employees safe during winter ice storms and -30°C temperatures? Through AI and accurate outage prediction models. ...read more


This man wants your next employee to be an AI-powered digital human

Written by Jordan Teicher | October 1, 2018 | Artificial Intelligence, Big Thinkers, Energy and Utilities...

Can digital humans revolutionize the economics of customer service? Greg Cross thinks so. ...read more