Trajectron++: Dynamically-Feasible Trajectory Forecasting With Heterogeneous Data
About
Reasoning about human motion is an important prerequisite to safe and socially-aware robotic navigation. As a result, multi-agent behavior prediction has become a core component of modern human-robot interactive systems, such as self-driving cars. While there exist many methods for trajectory forecasting, most do not enforce dynamic constraints and do not account for environmental information (e.g., maps). Towards this end, we present Trajectron++, a modular, graph-structured recurrent model that forecasts the trajectories of a general number of diverse agents while incorporating agent dynamics and heterogeneous data (e.g., semantic maps). Trajectron++ is designed to be tightly integrated with robotic planning and control frameworks; for example, it can produce predictions that are optionally conditioned on ego-agent motion plans. We demonstrate its performance on several challenging real-world trajectory forecasting datasets, outperforming a wide array of state-of-the-art deterministic and generative methods.
Key insights
What trajectory forecasting problem does this solve?
It predicts multi-agent trajectories while enforcing dynamic feasibility and using heterogeneous data like semantic maps. It outputs control actions integrated through agent dynamics, optionally conditioned on ego-robot motion plans.
How does it differ from prior trajectory methods?
Unlike predecessors ignoring dynamics, maps, or ego-motion, it combines all five desiderata. On ETH/UCY Best-of-20, it achieves 55-60% lower average ADE/FDE than MATF, the previous best generative baseline.
What practical use does this research serve?
It is designed for integration with robotic planning and control, producing full distributions useful for safer downstream decisions. The paper documents benchmark evaluation, not real-world deployment.
What limitations do the authors note?
The model was trained to predict only 3 seconds ahead, so 4-second results test generalization beyond training. Simplified dynamics models are used, and no public models exist for some baselines, limiting comparison.