# VFusion3D: Learning Scalable 3D Generative Models from Video Diffusion Models

Source: [VFusion3D: Learning Scalable 3D Generative Models from Video Diffusion Models](<https://www.sota2.com/research/paper/68fa74321e209131cea605a6>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T03:04:50.888Z

Authors: Junlin Han, Filippos Kokkinos, Philip Torr

Published: 2024-03-18T17:59:12.000Z

## Abstract

This paper presents a novel method for building scalable 3D generative models utilizing pre-trained video diffusion models. The primary obstacle in developing foundation 3D generative models is the limited availability of 3D data. Unlike images, texts, or videos, 3D data are not readily accessible and are difficult to acquire. This results in a significant disparity in scale compared to the vast quantities of other types of data. To address this issue, we propose using a video diffusion model, trained with extensive volumes of text, images, and videos, as a knowledge source for 3D data. By unlocking its multi-view generative capabilities through fine-tuning, we generate a large-scale synthetic multi-view dataset to train a feed-forward 3D generative model. The proposed model, VFusion3D, trained on nearly 3M synthetic multi-view data, can generate a 3D asset from a single image in seconds and achieves superior performance when compared to current SOTA feed-forward 3D generative models, with users preferring our results over 90% of the time.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Single Image to 3D Reconstruction on Google Scanned Objects (GSO) orbiting views](<https://www.sota2.com/research/sota/single-image-to-3d-reconstruction-on-google-scanned-objects-gso-orbiting-views.md>): Single Image to 3D Reconstruction; Google Scanned Objects (GSO) orbiting views; PSNR: 17.416.

- [Text-to-Object Generation on DreamFusion prompts](<https://www.sota2.com/research/sota/text-to-object-generation-on-dreamfusion-prompts.md>): Text-to-Object Generation; DreamFusion prompts; CLIP Image-Text Similarity: 0.265.

- [Single-image to 3D Reconstruction on Google Scanned Objects (GSO) orbiting views (CD, F-Score)](<https://www.sota2.com/research/sota/single-image-to-3d-reconstruction-on-google-scanned-objects-gso-orbiting-views-cd-f-score.md>): Single Image to 3D Reconstruction; Google Scanned Objects (GSO) orbiting views; Chamfer Distance: 0.16118.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2403.12034>)

[PDF](<https://arxiv.org/pdf/2403.12034.pdf>)