# Decoding Pedestrian Crossing Intention from Egocentric Vision via Vision Language Models

Source: [Decoding Pedestrian Crossing Intention from Egocentric Vision via Vision Language Models](<https://www.sota2.com/research/paper/6a279e5490966ade919bf9b8>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-10T05:15:22.625Z

Authors: Danya Li, Xiang Su, Yan Feng, Rico Krueger

Published: 2026-06-08T07:39:42.000Z

## Abstract

Egocentric vision offers a first-person view of human perception and decision making, yet its potential for traffic-safety prediction remains underexplored. In this work, we study the decoding of pedestrian crossing intentions from short egocentric video clips. We approach this by formulating the task as a closed-ended visual question answering (VQA) problem and leveraging vision language models (VLMs) to predict the pedestrians' intent. We first benchmark three families of state-of-the-art VLMs in a zero-shot setting, finding that they achieve moderate gains over random guessing but exhibit limited higher-level traffic reasoning. Motivated by these findings, we further adapt VLMs to the target task using parameter-efficient fine-tuning. Our results show that the fine-tuned models substantially outperform their zero-shot counterparts and achieve a 9\\% accuracy improvement over a specialized transformer-based baseline. Finally, we demonstrate that incorporating additional contextual cues, including ego motion, vehicle motion, and eye gaze, further improves predictive performance. In particular, the fine-tuned Qwen3-VL-2B model guided by eye gaze and ego motion achieves a 14.5% accuracy improvement over the transformer baseline, establishing a new state of the art for egocentric pedestrian intent decoding.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Pedestrian crossing intention prediction on VR-based egocentric video dataset (test)](<https://www.sota2.com/research/sota/pedestrian-crossing-intention-prediction-on-vr-based-egocentric-video-dataset-test.md>): Pedestrian crossing intention prediction; VR-based egocentric video dataset (test); Accuracy: 77.7.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2606.09142>)

[PDF](<https://arxiv.org/pdf/2606.09142.pdf>)
