Information to Get from Here to There Safely
Stacy Doore is working toward making navigation more accessible for the vision-impaired
You’re attending a business conference in a city you don’t know. After the day’s sessions, some new friends invite you to join them for dinner at a local restaurant. You’re not familiar with this restaurant. No worries, they say; it’s just down the street from the hotel. You can’t miss it. We’ll meet you there.
For the 7.5 million people in the United States who are blind or have low vision (BLV), this type of independent navigation in an unfamiliar place is a difficult problem, even with an experienced guide dog and a typical navigation app such as Google Maps. These types of apps provide several route options based on time and distance, but do not provide critical information about the accessibility features of the routes.
Stacy Doore, assistant professor of computer science, is working on a solution to this problem with funding from a National Institutes of Health multi-institution project. The five-year project, titled “VisionWay: Accessibility-aware Path Selection for Wayfinding,” is supported by a $2.7-million grant from the National Eye Institute. Doore is a member of an interdisciplinary research team that combines expertise in engineering, computer science, assistive technologies, and ophthalmology to develop a mobile accessible navigation framework to help people with visual impairments in wayfinding.
Marco Brocanelli from the Ohio State University is the principal investigator for this project, with Dr. Sayoko Moroi and Walter Wu, also from OSU; Nicholas A. Giudice from the University of Maine; Paul Fink, from Grand Valley State University; and Dongxiao Zhu, from Wayne State University. At Colby, Doore will focus on the use of spatial language and training the VisionWay system to deliver effective non-visual navigation descriptions with personalized route accessibility information.
Navigation safety is the goal
The goal of VisionWay is to allow a BLV navigator to evaluate the safety of navigation routes by incorporating multimodal vibro-audio feedback and detailed spatial language directions regarding the accessibility of pedestrian paths. It will also provide a way for BLV navigators to review and learn “pre-journey” information about the route they choose as the best one based on their personalized needs and preferences saved in the user’s profile.
Developed for use on a smartphone, the AI-based system will provide the user with a choice of three navigation routes to a destination. Each route will include an accessibility score based on crucial information such as distance, inclines, obstacles, types of street crossings, work zones, and so forth.
In addition, the VisionWay system will guide BLV navigators to follow a route on the screen with their finger, being notified in advance where obstacles are located and receiving other touch-based spatial information. The system “provides important contextual information about the environment through multiple sensory interactions,” said Doore, “because there are many ways we can process and communicate spatial information.” She noted that several of the researchers developing VisionWay “are experts in non-visual navigation theory and lived experience, this is a strength of this project.”
The ultimate goal of VisionWay is to help people travel in unfamiliar places safely and independently. The system builds on what she calls “ground truths”—critical human-identified spatial information combined with current street-view data.
“There are many more sources of verified spatial data available for the system to draw upon for training than even five years ago. What we want to do is to give the navigator a mental model of a route based on all of that information and to provide it in a way that’s useful and user-friendly,” Doore said. “Navigating unfamiliar places is very difficult for blind and low-vision navigators. We want the system to reduce the cognitive load for the navigator and allow them to make decisions about the routes before they begin their journey that best matches their abilities and wayfinding preferences.”
Expanded uses and opportunities
While the target group for this stage of research is for BLV navigators, the system can be expanded to personalize the information someone with mobility concerns would need to evaluate the accessibility of different navigation routes based on their needs and preferences.
Doore’s primary scholarship interests are centered on understanding how humans and machines process, communicate, and act upon spatial information about the world. This includes the optimization of spatial information to aid in non-visual perception, learning, and navigation using multimodal interfaces. Designing these interfaces involves working with BLV co-designers to create complex sets of human-computer interactions using dynamic positioning sensor data, computer vision, and generative AI models.
Doore became interested in emerging assistive technologies as a part of her doctoral and post-doctoral work. She established her own assistive technologies lab, the INSITE Lab at Colby, focusing on spatial information access, when she joined Colby’s Computer Science Department in 2020.
The INSITE lab typically has 10 to 15 undergraduate research assistants working on several interdisciplinary research projects with her Colby colleagues using AI, robotics, and mixed-reality technologies. She has developed other assistive navigation applications, some of which have won national recognition from the U.S. Department of Transportation. That project and several others helped to lay the groundwork for VisionWay, based on the ongoing spatial language information research she and her students are conducting, supported through other federal grants.
Many of her former research assistants are completing graduate school programs in human-computer interaction, robotics, and AI; others are employed in industry AI-governance positions. Doore believes that Colby students benefit from authentic undergraduate research opportunities, such as VisionWay, where they learn firsthand about both the technical skills needed for conducting assistive technology research and responsible computing research practices. They also have opportunities to attend academic conferences and present work they have coauthored with her.
An additional NSF grant
Besides VisionWay, Doore is also investigating a technology-based solution that improves the accessibility of spreadsheet applications for blind and low-vision data professionals. This research is part of a three-year National Science Foundation Collaboration Research grant with Northeastern University and the University of Maine.
Data tasks (i.e., data exploration, manipulation, and analysis) are often inaccessible to BLV professionals due to the visual biases of commonly used data tools and the limitations of current keyboard-based functions. During the first year of the grant, Doore and her students interviewed 50 data professionals to inform interface design features and guidelines for developing natural-language interactions to facilitate non-visual data analytics. “We are also partnering with BLV professionals and researchers who will be helping to co-design the application through workshops, prototyping evaluation, and experimental testing,” she said.
She and her students are also investigating how a large quadruped robot and its built-in navigation system may serve as a guide for a BLV handler. This project allows Doore to ask questions about the robot’s ability to accurately capture, interpret, and respond to the environment and provide spatial information about its surroundings to its handler. The team is developing a multimodal AI-based interface (images, audio, and kinesthetic) to facilitate communication of spatial information between the human-robot team and using reinforcement learning methods to improve the robot’s object-detection and autonomous-navigation capabilities.
“This summer, we implemented NVIDIA’s IsaacSim software to train a simulated version of our robot to learn how to navigate complex environments and public spaces,” she said. “The social component of a large, industrial robot operating as a navigation assistant in dynamic public spaces is a significant challenge for the eventual use-case application of this technology. We are investigating how to best establish new social norms around human-robot teams in public spaces to help signal the intentions and needs of an assistive navigation guide team.”
