# OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generation

Source: [OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generation](<https://www.sota2.com/research/paper/68fa76f41e209131cea703bc>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T04:07:06.968Z

Authors: Kepan Nan, Rui Xie, Penghao Zhou, Tiehan Fan, Zhenheng Yang, Zhijie Chen, Xiang Li, Jian Yang, Ying Tai

Published: 2024-07-02T15:40:29.000Z

## Abstract

Text-to-video (T2V) generation has recently garnered significant attention thanks to the large multi-modality model Sora. However, T2V generation still faces two important challenges: 1) Lacking a precise open sourced high-quality dataset. The previous popular video datasets, e.g. WebVid-10M and Panda-70M, are either with low quality or too large for most research institutions. Therefore, it is challenging but crucial to collect a precise high-quality text-video pairs for T2V generation. 2) Ignoring to fully utilize textual information. Recent T2V methods have focused on vision transformers, using a simple cross attention module for video generation, which falls short of thoroughly extracting semantic information from text prompt. To address these issues, we introduce OpenVid-1M, a precise high-quality dataset with expressive captions. This open-scenario dataset contains over 1 million text-video pairs, facilitating research on T2V generation. Furthermore, we curate 433K 1080p videos from OpenVid-1M to create OpenVidHD-0.4M, advancing high-definition video generation. Additionally, we propose a novel Multi-modal Video Diffusion Transformer (MVDiT) capable of mining both structure information from visual tokens and semantic information from text tokens. Extensive experiments and ablation studies verify the superiority of OpenVid-1M over previous datasets and the effectiveness of our MVDiT.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Video Reconstruction on Reconstruction-via-recaption](<https://www.sota2.com/research/sota/video-reconstruction-on-reconstruction-via-recaption.md>): Video Reconstruction; Reconstruction-via-recaption; 3DVAEscore: 139.88.

- [Text-to-Video Generation on Inseval (test)](<https://www.sota2.com/research/sota/text-to-video-generation-on-inseval-test.md>): Text-to-Video Generation; Inseval (test); Single Action: 0.52.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2407.02371>)

[PDF](<https://arxiv.org/pdf/2407.02371.pdf>)
