HyVGGT-VO: Tightly Coupled Hybrid Dense Visual Odometry with Feed-Forward Models
About
Dense visual odometry (VO), which provides pose estimation and dense 3D reconstruction, serves as the cornerstone for applications ranging from robotics to augmented reality. Recently, feed-forward models have demonstrated remarkable capabilities in dense mapping. However, when these models are used in dense visual SLAM systems, their heavy computational burden restricts them to yielding sparse pose outputs at keyframes while still failing to achieve real-time pose estimation. In contrast, traditional sparse methods provide high computational efficiency and high-frequency pose outputs, but lack the capability for dense reconstruction. To address these limitations, we propose HyVGGT-VO, a novel framework that combines the computational efficiency of sparse VO with the dense reconstruction capabilities of feed-forward models. To the best of our knowledge, this is the first work to tightly couple a traditional VO framework with VGGT, a state-of-the-art feed-forward model. Specifically, we design an adaptive hybrid tracking frontend that dynamically switches between traditional optical flow and the VGGT tracking head to ensure robustness. Furthermore, we introduce a hierarchical optimization framework that jointly refines VO poses and the scale of VGGT predictions to ensure global scale consistency. Our approach achieves an approximately 5x processing speedup compared to existing VGGT-based methods, while reducing the average trajectory error by 85% on the indoor EuRoC dataset and 12% on the outdoor KITTI benchmark. Our code will be publicly available upon acceptance. Project page: https://geneta2580.github.io/HyVGGT-VO.io.
Key insights
What problem does HyVGGT-VO solve?
Feed-forward models like VGGT enable dense 3D mapping but are too slow for real-time pose estimation and suffer scale drift. HyVGGT-VO tightly couples efficient sparse visual odometry with VGGT for real-time, globally consistent dense reconstruction.
How does it differ from prior VGGT methods?
Unlike VGGT-SLAM 2.0, which yields sparse poses at 3.3 FPS on an RTX 4090, HyVGGT-VO runs at 16.1 FPS on an RTX 4060. It reduces average trajectory error by 85% on EuRoC and 12% on KITTI.
What are the practical deployment uses?
The system provides high-frequency poses and dense maps on a mobile laptop, making it suitable for real-time robotics and augmented reality. The paper documents a benchmark prototype, not an actual production deployment.
What limitations do the authors acknowledge?
As a pure monocular VO framework without loop closures, it suffers severe scale drift on exceptionally long driving trajectories, as observed on specific KITTI sequences. The design is extensible to similar feed-forward models.