# Perceive, Interact, Predict: Learning Dynamic and Static Clues for End-to-End Motion Prediction

Source: [Perceive, Interact, Predict: Learning Dynamic and Static Clues for End-to-End Motion Prediction](<https://www.sota2.com/research/paper/68fa6e1d1e209131cea3de22>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T00:50:11.375Z

Authors: Bo Jiang, Shaoyu Chen, Xinggang Wang, Bencheng Liao, Tianheng Cheng, Jiajie Chen, Helong Zhou, Qian Zhang, Wenyu Liu, Chang Huang

Published: 2022-12-05T11:37:41.000Z

## Abstract

Motion prediction is highly relevant to the perception of dynamic objects and static map elements in the scenarios of autonomous driving. In this work, we propose PIP, the first end-to-end Transformer-based framework which jointly and interactively performs online mapping, object detection and motion prediction. PIP leverages map queries, agent queries and mode queries to encode the instance-wise information of map elements, agents and motion intentions, respectively. Based on the unified query representation, a differentiable multi-task interaction scheme is proposed to exploit the correlation between perception and prediction. Even without human-annotated HD map or agent's historical tracking trajectory as guidance information, PIP realizes end-to-end multi-agent motion prediction and achieves better performance than tracking-based and HD-map-based methods. PIP provides comprehensive high-level information of the driving scene (vectorized static map and dynamic objects with motion information), and contributes to the downstream planning and control. Code and models will be released for facilitating further research.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Trajectory Forecasting on nuScenes (val)](<https://www.sota2.com/research/sota/trajectory-forecasting-on-nuscenes-val.md>): Trajectory Forecasting; nuScenes (val); EPA: 0.258.

- [3D Object Detection on nuScenes (val)](<https://www.sota2.com/research/sota/3d-object-detection-on-nuscenes-val.md>): 3D Object Detection; nuScenes (val); NDS: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2212.02181>)

[PDF](<https://arxiv.org/pdf/2212.02181.pdf>)