Two new applications are set to assist visually impaired individuals in navigating indoor spaces with verbal guidance from their smartphones, offering a reliable way to find their way when GPS is ineffective.
Roberto Manduchi, a professor of Computer Science and Engineering at UC Santa Cruz, has spent a significant part of his career dedicated to developing technology that is accessible for individuals who are blind or have low vision. His extensive work with these communities has highlighted the urgent need for tools that aid in navigating unfamiliar indoor environments.
“Getting around independently in an unfamiliar area is especially challenging because there are no visual cues — it’s easy to become disoriented. This initiative aims to make it a bit easier and safer for users,” Manduchi explained.
In a recent study published in the ACM Transactions on Accessible Computing, Manduchi’s team introduced two smartphone applications designed to assist with indoor wayfinding, pinpoint navigation to designated locations, and safe route retracing. The applications provide audio instructions and don’t require users to hold their phones in front of them, which would be awkward and could draw unwanted attention.
Enhanced, scalable technology
Smartphones are an ideal platform for accessible technology because they are more affordable than dedicated devices, benefit from robust IT support, and come equipped with built-in sensors and accessibility options.
Many other smartphone-based navigation systems necessitate that the user keeps their phone visible, which presents several challenges. A blind individual managing their way through a new environment typically has at least one hand occupied with a guide dog or cane, making it less practical to use the other hand for a phone. Furthermore, holding a phone out in public can increase vulnerability to crime, an issue that disproportionately affects individuals with disabilities.
While tech giants like Apple and Google have implemented indoor navigation in select locations such as large airports and stadiums, their solutions are reliant on specialized sensors placed inside these venues, making them less scalable due to the high costs associated with installing and maintaining such infrastructure.
Leveraging built-in sensors
Manduchi’s navigation application functions similarly to GPS services like Google Maps, but GPS technology fails indoors as the satellite signals are obstructed by building structures. Instead, his system utilizes the smartphone’s internal sensors to deliver verbal directions for navigating through unfamiliar facilities.
The app operates by mapping the interior of a building to determine a route to the user’s desired destination and then employs the phone’s built-in inertial sensors, accelerometers, and gyros — which support features like step counting — to monitor the user’s journey along the designated path.
These sensors also track the orientation of the phone and, by extension, the individual using it. However, since the perceived location and orientation may not always be precise, the researchers employed a technique called particle filtering to ensure the app adheres to the physical boundaries of the building, preventing it from inaccurately suggesting paths through walls or other obstacles.
The backtracking app enables users to retrace their steps by reversing the route they previously followed. This feature is especially useful in situations where a blind person is escorted into a room and wishes to exit independently. In addition to utilizing inertial sensors, this app leverages the phone’s magnetometer to identify unique magnetic field variations typically generated by large appliances, which can act as navigational landmarks within a building.
Providing directions
Both applications communicate directions through audio and can also connect to smartwatches, which can provide additional guidance through vibrations. The researchers aimed to minimize the amount of information relayed to users, allowing them to prioritize their safety.
Users are also expected to make their own decisions regarding when to turn, accommodating any potential tracking errors. The system prompts the user for their next directional change five meters in advance, with instructions like “at the next junction, turn left,” allowing the navigator to locate the turn with the assistance of their cane or guide dog.
“I believe in the importance of sharing responsibility,” Manduchi stated. “Just as you wouldn’t rely solely on navigation tools while driving—if it indicates a right turn, you still check for the junction—you have to interact with the system.”
During testing of their systems in UC Santa Cruz’s Baskin Engineering building, the research team found that users successfully navigated the myriad hallways and turns. The team plans to refine their applications, which share the same interface but remain distinct for developmental ease.
As they move forward, they will aim to incorporate AI functionalities that enable users to take photos of their surroundings and receive descriptions of their environment, particularly in challenging navigation scenarios like alcoves or open areas. They are also looking to improve the process of accessing and downloading building maps, potentially utilizing an open-source software framework to facilitate this.
“I’m incredibly thankful to the blind community in Santa Cruz for their invaluable insights. When engineers create technology intended for the blind community, it’s crucial to approach the task with utmost care and humility, focusing first on the individual’s needs rather than the technology itself,” Manduchi remarked.
