# NaviSense: A Multimodal Assistive Mobile application for Object Retrieval by Persons with Visual Impairment

Source: [NaviSense: A Multimodal Assistive Mobile application for Object Retrieval by Persons with Visual Impairment](<https://www.sota2.com/research/paper/68fa7b801e209131cea88b8e>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T05:41:29.127Z

Authors: Ajay Narayanan Sridhar, Fuli Qiao, Nelson Daniel Troncoso Aldas, Yanpei Shi, Mehrdad Mahdavi, Laurent Itti, Vijaykrishnan Narayanan (1) \_\_INSTITUTION\_7\_\_ The Pennsylvania State University, (2) Independent Researcher, (3) University of Southern California)

Published: 2025-09-23T05:45:11.000Z

## Abstract

People with visual impairments often face significant challenges in locating and retrieving objects in their surroundings. Existing assistive technologies present a trade-off: systems that offer precise guidance typically require pre-scanning or support only fixed object categories, while those with open-world object recognition lack spatial feedback for reaching the object. To address this gap, we introduce 'NaviSense', a mobile assistive system that combines conversational AI, vision-language models, augmented reality (AR), and LiDAR to support open-world object detection with real-time audio-haptic guidance. Users specify objects via natural language and receive continuous spatial feedback to navigate toward the target without needing prior setup. Designed with insights from a formative study and evaluated with 12 blind and low-vision participants, NaviSense significantly reduced object retrieval time and was preferred over existing tools, demonstrating the value of integrating open-world perception with precise, accessible guidance.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2509.18672>)

[PDF](<https://arxiv.org/pdf/2509.18672.pdf>)
