# ViLAM: Distilling Vision-Language Reasoning into Attention Maps for Social Robot Navigation

Source: [ViLAM: Distilling Vision-Language Reasoning into Attention Maps for Social Robot Navigation](<https://www.sota2.com/research/paper/68fa77621e209131cea72dd8>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-11T05:00:04.181Z

Authors: Mohamed Elnoor, Kasun Weerakoon, Gershom Seneviratne, Jing Liang, Vignesh Rajagopal, Dinesh Manocha

Published: 2025-03-12T20:38:23.000Z

## Abstract

We introduce ViLAM, a novel method for distilling vision-language reasoning from large Vision-Language Models (VLMs) into spatial attention maps for socially compliant robot navigation. Unlike traditional methods that rely on expert demonstrations or human-annotated datasets, ViLAM performs knowledge distillation and fine-tuning at the intermediate layer representation (attention) level by aligning attention maps from a pretrained vision-action model with socially guided attention maps derived from a large VLM. These distilled attention maps highlight key navigational regions in a scene and serve as socially informed spatial cost maps for motion planning. To achieve this, we introduce a novel attention-level distillation loss that fuses knowledge from both sources, generating augmented attention maps with enhanced social awareness. These refined attention maps are then used as a traversability costmap within a socially aware local planner for navigation. We validate our approach through real-world experiments on a Husky wheeled robot, and demonstrate 14.2% - 50% improvements in success rate over existing methods.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Robot Navigation on Husky Robot Navigation Scenario 1](<https://www.sota2.com/research/sota/robot-navigation-on-husky-robot-navigation-scenario-1.md>): Robot navigation; Husky Robot Navigation Scenario 1; Success Rate: 100.

- [Robot Navigation on Husky Robot Navigation (Scenario 2)](<https://www.sota2.com/research/sota/robot-navigation-on-husky-robot-navigation-scenario-2.md>): Robot navigation; Husky Robot Navigation (Scenario 2); Success Rate: 90.

- [Robot Navigation on Husky Robot Navigation (Scenario 3)](<https://www.sota2.com/research/sota/robot-navigation-on-husky-robot-navigation-scenario-3.md>): Robot navigation; Husky Robot Navigation (Scenario 3); Success Rate: 90.

- [Robot Navigation on Husky Robot Navigation (Scenario 4)](<https://www.sota2.com/research/sota/robot-navigation-on-husky-robot-navigation-scenario-4.md>): Robot navigation; Husky Robot Navigation (Scenario 4); Success Rate: 80.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2503.09820>)

[PDF](<https://arxiv.org/pdf/2503.09820.pdf>)
