Social distancing has become the norm — but for visually impaired people, adhering to the regulations can be tricky. Especially when it comes to standing in line.
Our team wants to help. We have developed an AI-driven assistive smartphone app dubbed LineChaser, presented at CHI 2021¹, that navigates a blind or visually impaired person to the end of a line. It also continuously reports the distance and direction to the last person in the line, so that the blind user can follow them easily.
Relying on the phone’s RGB camera, LineChaser detects nearby pedestrians, and the built-in infrared depth sensor estimates their position. These estimates help the app to determine whether nearby pedestrians are standing in line, and audio and vibration signals then notify the user when they should start or stop moving forward.
This is not the first time IBM Research has tried to help visually impaired people, many of whom still struggle with navigation in public places. Our researchers have in the past developed smartphone-based navigation apps.
Stand in line assistance
We tackled standing in line assistance in the past, too. In 2020, we relied on LiDAR and RGB-D camera devices to create AI Suitcase — a suitcase-style navigation robot that helps the visually impaired to move around the city independently including stand-in-lines. In this study, we decided to enable similar assistance with the help of an off-the-shelf smartphone.
Although the trial participants who tested LineChaser had various visual difficulties, the app successfully enabled everyone to both find and follow a line. They all successfully found the end of a line at appropriate positions, indicating that LineChaser can navigate a blind user properly to the end of a line.
LineChaser was also able to help blind users stop at acceptable positions 91.7 percent of the time while following a line. Feedback from the participants supported the effectiveness of our system with high usability score. All participants significantly increased most of their scores for confidence and comfortableness when standing in line.
We hope that more visually impaired people can benefit from our technology. We also hope that with time, the social acceptance of the use of camera devices in public spaces will improve. We found the acceptance was already high if a blind person used a device for assistive purposes, but we think that as a society, we need to improve our understanding of such technologies.
IBM Research-Tokyo interns and Waseda University students Masaki Kuribayashi and Seita Kayukawa also contributed to this article.
IBM Fellow and IBM Distinguished Service Professor at Carnegie Mellon University Chieko Asakawa will deliver the CHI 2021 keynote: See What I Mean: Making Waves with the Blind.
Our study "Comparison of methods to reduce bias from clinical prediction models of postpartum depression” examines healthcare data and machine learning models routinely used in both research and application to address bias in healthcare AI.
Founded in March 2020 just as the pandemic’s wave was starting to wash over the world, the Consortium has brought together 43 members with supercomputing resources. Private and public enterprises, academia, government and technology companies, many of whom are typically rivals. “It is simply unprecedented,” said Dario Gil, Senior Vice President and Director of IBM Research, one of the founding organizations. “The outcomes we’ve achieved, the lessons we’ve learned, and the next steps we have to pursue are all the result of the collective efforts of these Consortium’s community.”
The next step? Creating the National Strategic Computing Reserve to help the world be better prepared for future global emergencies.