VGGT-$\Omega$
About
Recent feed-forward reconstruction models, such as VGGT, have proven competitive with traditional optimization-based reconstructors while also providing geometry-aware features useful for other tasks. Here, we show that the quality of these models scales predictably with model and data size. We do so by introducing VGGT-$\Omega$, which substantially improves reconstruction accuracy, efficiency, and capabilities for both static and dynamic scenes. To enable training this model at an unprecedented scale, we introduce architectural changes that improve training efficiency, a high-quality data annotation pipeline that supports dynamic scenes, and a self-supervised learning protocol. We simplify VGGT's architecture by using a single dense prediction head with multi-task supervision and removing the expensive high-resolution convolutional layers. We also use registers to aggregate scene information into a compact representation and introduce register attention, which restricts inter-frame information exchange to these registers, in part replacing global attention. In this way, during training, VGGT-$\Omega$ uses only about 30% of the GPU memory of its predecessor, allowing us to train with 15x more supervised data than prior work and to leverage vast amounts of unlabeled video data. VGGT-$\Omega$ achieves strong results for reconstruction of static and dynamic scenes across multiple benchmarks, for example, improving over the previous best camera estimation accuracy on Sintel by 77%. We also show that the learned registers can improve vision-language-action models and support alignment with language, suggesting that reconstruction can be a powerful and scalable proxy task for spatial understanding. Project Page: http://vggt-omega.github.io/
Key insights
What problem does VGGT-Ω solve?
It studies whether feed-forward 3D reconstruction models scale with model and data size. It introduces register attention, a single dense head, and self-supervised training to cut memory 70%, enabling 15× more supervised data and dynamic-scene support.
How does it differ from prior work?
Unlike VGGT, it replaces 25% of global attention with register attention (no measurable drop, 23% FLOP savings) and uses 4M sequences vs. VGGT's fewer. On Sintel, AUC@3° improves from 22.5 to 40.0 (77%) over MegaSaM.
What are the practical uses?
Demonstrated on six static and dynamic benchmarks, it also shows registers improve VLA models on LIBERO and align with language via contrastive learning.
What limitations do the authors note?
Performance drops with strong motion blur, abrupt field-of-view changes, or high distortion. Noisy early-training data (e.g., ScanNet++) causes instability in office scenes with monitors. MLP-only decoders produce blocky artifacts, so shallow convolutions are retained.