# Uni4D: Unifying Visual Foundation Models for 4D Modeling from a Single Video

Source: [Uni4D: Unifying Visual Foundation Models for 4D Modeling from a Single Video](<https://www.sota2.com/research/paper/68fa779d1e209131cea745c6>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T04:23:33.312Z

Authors: David Yifan Yao, Albert J. Zhai, Shenlong Wang

Published: 2025-03-27T17:57:32.000Z

## Abstract

This paper presents a unified approach to understanding dynamic scenes from casual videos. Large pretrained vision foundation models, such as vision-language, video depth prediction, motion tracking, and segmentation models, offer promising capabilities. However, training a single model for comprehensive 4D understanding remains challenging. We introduce Uni4D, a multi-stage optimization framework that harnesses multiple pretrained models to advance dynamic 3D modeling, including static/dynamic reconstruction, camera pose estimation, and dense 3D motion tracking. Our results show state-of-the-art performance in dynamic 4D modeling with superior visual quality. Notably, Uni4D requires no retraining or fine-tuning, highlighting the effectiveness of repurposing visual foundation models for 4D understanding.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Pose Estimation on Bonn](<https://www.sota2.com/research/sota/pose-estimation-on-bonn.md>): Pose Estimation; BONN; ATE: 0.017.

- [Camera Pose Estimation on TUM dynamics](<https://www.sota2.com/research/sota/camera-pose-estimation-on-tum-dynamics.md>): Camera pose estimation; TUM dynamics; ATE: 0.039.

- [Camera Pose Estimation on Sintel](<https://www.sota2.com/research/sota/camera-pose-estimation-on-sintel.md>): Camera pose estimation; Sintel; ATE: 0.116.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2503.21761>)

[PDF](<https://arxiv.org/pdf/2503.21761.pdf>)
