# Open3DTrack: Towards Open-Vocabulary 3D Multi-Object Tracking

Source: [Open3DTrack: Towards Open-Vocabulary 3D Multi-Object Tracking](<https://www.sota2.com/research/paper/68fa772c1e209131cea71921>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T04:11:54.790Z

Authors: Ayesha Ishaq, Mohamed El Amine Boudjoghra, Jean Lahoud, Fahad Shahbaz Khan, Salman Khan, Hisham Cholakkal, Rao Muhammad Anwer

Published: 2024-10-02T15:48:42.000Z

## Abstract

3D multi-object tracking plays a critical role in autonomous driving by enabling the real-time monitoring and prediction of multiple objects' movements. Traditional 3D tracking systems are typically constrained by predefined object categories, limiting their adaptability to novel, unseen objects in dynamic environments. To address this limitation, we introduce open-vocabulary 3D tracking, which extends the scope of 3D tracking to include objects beyond predefined categories. We formulate the problem of open-vocabulary 3D tracking and introduce dataset splits designed to represent various open-vocabulary scenarios. We propose a novel approach that integrates open-vocabulary capabilities into a 3D tracking framework, allowing for generalization to unseen object classes. Our method effectively reduces the performance gap between tracking known and novel objects through strategic adaptation. Experimental results demonstrate the robustness and adaptability of our method in diverse outdoor driving scenarios. To the best of our knowledge, this work is the first to address open-vocabulary 3D tracking, presenting a significant advancement for autonomous systems in real-world settings. Code, trained models, and dataset splits are available publicly.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [3D Multi-Object Tracking on V2X-Seq-SPD Vehicle Side](<https://www.sota2.com/research/sota/3d-multi-object-tracking-on-v2x-seq-spd-vehicle-side.md>): 3D Multi-Object Tracking; V2X-Seq-SPD Vehicle Side; sAMOTA: 29.13.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2410.01678>)

[PDF](<https://arxiv.org/pdf/2410.01678.pdf>)
