TransTrack: Multiple Object Tracking with Transformer
About
In this work, we propose TransTrack, a simple but efficient scheme to solve the multiple object tracking problems. TransTrack leverages the transformer architecture, which is an attention-based query-key mechanism. It applies object features from the previous frame as a query of the current frame and introduces a set of learned object queries to enable detecting new-coming objects. It builds up a novel joint-detection-and-tracking paradigm by accomplishing object detection and object association in a single shot, simplifying complicated multi-step settings in tracking-by-detection methods. On MOT17 and MOT20 benchmark, TransTrack achieves 74.5\% and 64.5\% MOTA, respectively, competitive to the state-of-the-art methods. We expect TransTrack to provide a novel perspective for multiple object tracking. The code is available at: \url{https://github.com/PeizeSun/TransTrack}.
Key insights
What tracking problem does TransTrack solve?
Multiple object tracking (MOT) traditionally uses multi-stage pipelines where detection and association are separate. TransTrack uses a transformer to jointly detect objects and propagate them across frames in a single shot, using two query types.
How does TransTrack differ from prior MOT methods?
Unlike tracking-by-detection or single-query SOT approaches that miss new objects, TransTrack uses both learned object queries for new detections and track queries from prior-frame features. It achieves 74.5% MOTA on MOT17 and 64.5% on MOT20.
What is TransTrack useful for in practice?
TransTrack provides a transformer-based baseline for pedestrian tracking in crowded video scenes. The paper documents benchmark evaluation on MOT17 and MOT20 datasets but does not document any production deployment.
What limitations do the authors acknowledge?
The authors note TransTrack's ID-switch performance is inferior to state-of-the-art methods, suggesting it as a direction for improvement. Evaluation is limited to pedestrian benchmarks MOT17 and MOT20.