# ForeSight: Multi-View Streaming Joint Object Detection and Trajectory Forecasting

Source: [ForeSight: Multi-View Streaming Joint Object Detection and Trajectory Forecasting](<https://www.sota2.com/research/paper/68fa7a801e209131cea8384e>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T05:22:00.744Z

Authors: Sandro Papais, Letian Wang, Brian Cheong, Steven L. Waslander

Published: 2025-08-09T20:18:10.000Z

## Abstract

We introduce ForeSight, a novel joint detection and forecasting framework for vision-based 3D perception in autonomous vehicles. Traditional approaches treat detection and forecasting as separate sequential tasks, limiting their ability to leverage temporal cues. ForeSight addresses this limitation with a multi-task streaming and bidirectional learning approach, allowing detection and forecasting to share query memory and propagate information seamlessly. The forecast-aware detection transformer enhances spatial reasoning by integrating trajectory predictions from a multiple hypothesis forecast memory queue, while the streaming forecast transformer improves temporal consistency using past forecasts and refined detections. Unlike tracking-based methods, ForeSight eliminates the need for explicit object association, reducing error propagation with a tracking-free model that efficiently scales across multi-frame sequences. Experiments on the nuScenes dataset show that ForeSight achieves state-of-the-art performance, achieving an EPA of 54.9%, surpassing previous methods by 9.3%, while also attaining the best mAP and minADE among multi-view detection and forecasting models.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Trajectory Forecasting on nuScenes (val)](<https://www.sota2.com/research/sota/trajectory-forecasting-on-nuscenes-val.md>): Trajectory Forecasting; nuScenes (val); EPA: 0.499.

- [3D Object Detection on nuScenes (val)](<https://www.sota2.com/research/sota/3d-object-detection-on-nuscenes-val.md>): 3D Object Detection; nuScenes (val); NDS: 58.9.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2508.07089>)

[PDF](<https://arxiv.org/pdf/2508.07089.pdf>)