How an app designed for people who are blind could solve a global skills shortage

By | 3 minute read | October 24, 2018

Kilroy Blockchain's “Riley (RealLife Adventure)" app was originally designed to overcome sight-related barriers.

When Karen Kilroy volunteered to coach the dragon boat racing team at the Texas School for the Blind and Visually Impaired (TSBVI), she didn’t expect to come out of it with an idea for a new application for AI—and a way to help solve a growing problem in the global talent pool. But that’s what happened one day last year when the Austin-based CEO of Kilroy Blockchain was leading a group of students in practice for the team sport, which requires crews of 20 paddlers to propel 40-foot-long boats on open water.

In between bouts on the water, Kilroy noticed that the students were spending a lot of down time on their phones. What if, she wondered, an app could allow them to use those devices to explore the world beyond their screens? “This seemed really exciting to me, because the right application would have the potential to open up a new world of interaction and information for the visually impaired. Back at the office, my team and I got to work on refining our solution idea,” she told IBM. Soon after, Kilroy learned about IBM’s inaugural Watson Build Challenge, a global initiative designed to spark the development of AI-based solutions. It was just the platform she needed to help turn her idea into a reality.

Along with more than a thousand other IBM business partners, Kilroy pitched her idea, an app called “Riley (RealLife Adventure),” to the global competition, and was ultimately selected as the winner for North America. The app, which officially launched at the Consumer Electronics Show in January, has two main features. One, “What’s That?” uses Watson Visual Recognition to identify objects in a user’s photo and audibly describes it through speech-to-text technology. The other, “Look Around,” uses Watson Content Hub to provide an audible description of a user’s surroundings based on their geolocation.

“We often train [visually impaired] students to use telescopes and binoculars to look at what’s around them, but it makes them kind of different and teenagers don’t want to look different. So being able to explore their surroundings using a program like Riley has a positive psychological effect, particularly on teenagers,” said TSBVI Superintendent Bill Daugherty.

This fall, select students at TSBVI will test that theory when they take Riley for a spin as part of a pilot program. Primarily, they’ll be looking to evaluate how the app can be used to help overcome sight-related barriers. But in the future, said Kilroy, the app could also be used to tackle a major barrier facing business across industries—particularly, a growing skills shortage. According to a recent report by the Bpifrance public investment bank, the Financial Times reports, nine out of 10 mid-sized companies are facing recruitment difficulties. The main recruitment issue, the companies report, is a lack of qualified workers. Riley, said Kilroy, could help close that gap.

According to Kilroy, Riley’s server can be configured to become a visual subject matter expert for any business’ products. It could, for instance, be trained to recognize and distinguish specific types of produce, equipment in a coffee bar, or parts on an assembly line. It could then be used to train workers on how best to interact with those products. “It’s actually less of a heavy lift, because Riley doesn’t have to be able to describe the whole universe— just a narrow slice of it,” said Amanda Lacy, a quality assurance technician for Riley who has been blind since birth. People who are blind and visually impaired are already using Riley to explore their world. Now, Kilroy said, the next big “real life adventure” awaits companies looking to take enable their workforces with a game-changing training tool.

“Maybe the limitation is sight, or maybe it’s language. Maybe the limitation is knowledge. What we’re looking for are companies that want to try out ‘new collar’ workforce tools by supplying their workforce with tools that enable people to do jobs they weren’t able to do in the past,” Kilroy said.