Share this post:
My company, Shimizu Corp., has long followed the management principles of Shibusawa Eiichi, who held that economic activity and morality are inseparable. This harmony can be seen in the experimental system we developed with IBM Research to help people, including those who are disabled, enjoy barrier-free urban exploration through voice navigation on their iPhones.
We conceived the indoor/outdoor system as a merger of the core construction technologies we have gathered over many years with new leading technologies like AI and robotics. The idea was to overcome the limitations of typical navigational apps that work poorly indoors because the infrastructure blocks GPS satellite signals.
As a field experiment, we installed our system in the COREDO Muromachi shopping mall, a 21,000-square-meter, three-building complex and underground walkway located in downtown Tokyo’s popular Nihonbashi-Muromachi district. The navigation would assist the general public, of course, but also people who are visually impaired, wheelchair users and those who don’t speak Japanese.
Teaming with our partners
We first gained the support of Mitsui Fudosan Co., Ltd., the mall’s real estate developer, which shared maps and architectural details. Our researchers used the information to create a spatial database about facilities needed for mall navigation, such as the location of entrances, stairways, halls, restrooms and elevators.
IBM Research contributed the NavCog navigational app. Developed with Carnegie Melon University, the app uses Bluetooth Low Energy (BLE) beacons for indoor positioning in concert with iPhone sensors. The BLE protocol is ideal for urban navigation due to its low cost, simplicity, accuracy and beacons located in interior spaces.
IBM Research also provided engineering support, machine learning expertise and AI services. The IBM Watson Assistant conversational computing platform powers voice dialogue in Japanese and English, and Watson Explorer provides a recommendation engine for the mall’s 100 tenants. Both services are delivered from the IBM Cloud.
Instrumentalizing the mall
Our researchers devised a unique method to mount the 224 BLE beacons needed to navigate the mall. By identifying locations in ceilings and narrow gaps that would avoid the need for renovations, they placed all the beacons in one night.
Next, IBM researchers used an electric wheelchair equipped with sensors to measure radio waves from the beacons without closing passageways or disturbing store operations. The work took just a few days. The IBM team then applied machine learning algorithms to the data to create a likelihood model to associate the BLE signals with pedestrian locations. We were quite pleased with the system’s accuracy: it can pinpoint a user’s location to within 1.5-meters average error, thought to be world’s most accurate indoor positioning to date.
Improving accessibility for all
The app features different operational modes for people who are visually impaired, wheelchair users and the general public in both English and Japanese.
If someone who is visually impaired says, “Take me to the movie theater,” for example, the iPhone shows a route map on the screen, but then quickly digests route information and voices detailed instructions by speaker, earbuds or bone-conducting headphone. The directions lay out the shortest route but can avoid escalators based on user preference. The system provides turn-by-turn navigation guidance to users, such as “Proceed 25 meters on tactile paving and turn left at warning tactile,” or “Proceed 6 meters and turn left at the end of corridor.” In an elevator, the system can explain “Go to the 3rd floor. Control buttons with braille are on the right side of the exit.”
Wheelchair users experience a different interface, either visual or voice, that presents directions avoiding staircases, steep slopes and facilities inaccessible by wheelchair.
The app’s recommendation engine easily responds to “I want to eat Chinese food” and other simple requests. But Watson Explorer can deeply mine mall data to handle more complex queries, such as “I want to take a date to a restaurant where good pasta and wine are available.” Taking cues from “pasta,” “wine available” and “date,” the recommendation engine would likely recommend an upscale Italian eatery.
Promising a better quality of life
Around 400 people tested the completed service, including 22 with weak eyesight and 21 who are totally blind. More than 70 percent rated the navigation “good” or “relatively good” in making the mall more accessible. As one person noted, “It is very helpful because I often get into trouble when I get to a building entrance.” Another said, “With this voice navigation system, I think I can move around a place when I visit it for the first time.”
In the words of Kazuyuki Inoue, Shimizu’s president, “Shimizu is dedicated to achieving sustained growth while creating environments in which people can live in comfort, safety and security.” We believe the navigation system furthers these goals through its potential to power a useful service for our customers and create a barrier-free urban environment for all.