# KV-Tracker: Real-Time Pose Tracking with Transformers

Source: [KV-Tracker: Real-Time Pose Tracking with Transformers](<https://www.sota2.com/research/paper/69535c8553fdd5c36173d996>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-12-31T05:00:03.851Z

Authors: Marwan Taher, Ignacio Alzugaray, Kirill Mazur, Xin Kong, Andrew J. Davison

Published: 2025-12-27T13:02:30.000Z

## Abstract

Multi-view 3D geometry networks offer a powerful prior but are prohibitively slow for real-time applications. We propose a novel way to adapt them for online use, enabling real-time 6-DoF pose tracking and online reconstruction of objects and scenes from monocular RGB videos. Our method rapidly selects and manages a set of images as keyframes to map a scene or object via $\\pi^3$ with full bidirectional attention. We then cache the global self-attention block's key-value (KV) pairs and use them as the sole scene representation for online tracking. This allows for up to $15\\times$ speedup during inference without the fear of drift or catastrophic forgetting. Our caching strategy is model-agnostic and can be applied to other off-the-shelf multi-view networks without retraining. We demonstrate KV-Tracker on both scene-level tracking and the more challenging task of on-the-fly object tracking and reconstruction without depth measurements or object priors. Experiments on the TUM RGB-D, 7-Scenes, Arctic and OnePose datasets show the strong performance of our system while maintaining high frame-rates up to ${\\sim}27$ FPS.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Object Tracking on Arctic Dataset](<https://www.sota2.com/research/sota/object-tracking-on-arctic-dataset.md>): Object Tracking; Arctic Dataset; ATE RMSE (m): 0.135.

- [Object Tracking on OnePose original (test)](<https://www.sota2.com/research/sota/object-tracking-on-onepose-original-test.md>): Object Tracking; OnePose original (test); Accuracy (1cm/1°): 10.7.

- [Object Tracking on OnePose Low Texture original (test)](<https://www.sota2.com/research/sota/object-tracking-on-onepose-low-texture-original-test.md>): Object Tracking; OnePose Low Texture original (test); Acc (1cm, 1°): 12.1.

- [Camera Tracking on TUM-RGBD (Specific Sequence Evaluation)](<https://www.sota2.com/research/sota/camera-tracking-on-tum-rgbd-specific-sequence-evaluation.md>): Camera Tracking; TUM-RGBD; Sequence 360 Error: 0.166.

- [Camera Localization on 7 Scenes](<https://www.sota2.com/research/sota/camera-localization-on-7-scenes.md>): Camera Localization; 7 Scenes; Average Position Error (m): 0.08.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2512.22581>)

[PDF](<https://arxiv.org/pdf/2512.22581.pdf>)