# UltraViCo: Breaking Extrapolation Limits in Video Diffusion Transformers

Source: [UltraViCo: Breaking Extrapolation Limits in Video Diffusion Transformers](<https://www.sota2.com/research/paper/6926899ad6b0dd3342930448>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-04T05:00:12.634Z

Authors: Min Zhao, Hongzhou Zhu, Yingze Wang, Bokai Yan, Jintao Zhang, Guande He, Ling Yang, Chongxuan Li, Jun Zhu

Published: 2025-11-25T09:44:10.000Z

## Abstract

Despite advances, video diffusion transformers still struggle to generalize beyond their training length, a challenge we term video length extrapolation. We identify two failure modes: model-specific periodic content repetition and a universal quality degradation. Prior works attempt to solve repetition via positional encodings, overlooking quality degradation and achieving only limited extrapolation. In this paper, we revisit this challenge from a more fundamental view: attention maps, which directly govern how context influences outputs. We identify that both failure modes arise from a unified cause: attention dispersion, where tokens beyond the training window dilute learned attention patterns. This leads to quality degradation and repetition emerges as a special case when this dispersion becomes structured into periodic attention patterns, induced by harmonic properties of positional encodings. Building on this insight, we propose UltraViCo, a training-free, plug-and-play method that suppresses attention for tokens beyond the training window via a constant decay factor. By jointly addressing both failure modes, we outperform a broad set of baselines largely across models and extrapolation ratios, pushing the extrapolation limit from 2x to 4x. Remarkably, it improves Dynamic Degree and Imaging Quality by 233% and 40.5% over the previous best method at 4x extrapolation. Furthermore, our method generalizes seamlessly to downstream tasks such as controllable video synthesis and editing.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Video Extrapolation on VBench Wan with 2x extrapolation v1.0 (test)](<https://www.sota2.com/research/sota/video-extrapolation-on-vbench-wan-with-2x-extrapolation-v1-0-test.md>): Video Extrapolation; VBench Wan with 2x extrapolation v1.0 (test); Dynamic Score: 68.

- [Video Extrapolation on VBench CogVideoX with 3x extrapolation v1.0 (test)](<https://www.sota2.com/research/sota/video-extrapolation-on-vbench-cogvideox-with-3x-extrapolation-v1-0-test.md>): Video Extrapolation; VBench CogVideoX with 3x extrapolation v1.0 (test); NoRepeat Score: 99.38.

- [Video Extrapolation on VBench HunyuanVideo with 5x extrapolation v1.0 (test)](<https://www.sota2.com/research/sota/video-extrapolation-on-vbench-hunyuanvideo-with-5x-extrapolation-v1-0-test.md>): Video Extrapolation; VBench HunyuanVideo with 5x extrapolation v1.0 (test); NoRepeat Score: 99.44.

- [Video Extrapolation on VBench CogVideoX with 2x extrapolation v1.0 (test)](<https://www.sota2.com/research/sota/video-extrapolation-on-vbench-cogvideox-with-2x-extrapolation-v1-0-test.md>): Video Extrapolation; VBench CogVideoX with 2x extrapolation v1.0 (test); NoRepeat Score: 98.92.

- [Video Extrapolation on VBench HunyuanVideo with 2x extrapolation v1.0 (test)](<https://www.sota2.com/research/sota/video-extrapolation-on-vbench-hunyuanvideo-with-2x-extrapolation-v1-0-test.md>): Video Extrapolation; VBench HunyuanVideo with 2x extrapolation v1.0 (test); NoRepeat Score: 97.53.

- [Video Extrapolation on VBench CogVideoX with 4x extrapolation v1.0 (test)](<https://www.sota2.com/research/sota/video-extrapolation-on-vbench-cogvideox-with-4x-extrapolation-v1-0-test.md>): Video Extrapolation; VBench CogVideoX with 4x extrapolation v1.0 (test); NoRepeat Score: 96.79.

- [Video Extrapolation on 5 long descriptive prompts 2x / 165f train-free extrapolation Wan 2.1 1.3B](<https://www.sota2.com/research/sota/video-extrapolation-on-5-long-descriptive-prompts-2x-165f-train-free-extrapolation-wan-2-1-1-3b.md>): Video Extrapolation; 5 long descriptive prompts 2x / 165f train-free extrapolation Wan 2.1 1.3B; VQeval: 62.1.

- [Video Extrapolation on 5 long descriptive prompts (3x / 249f) Wan 2.1 1.3B (train-free extrapolation)](<https://www.sota2.com/research/sota/video-extrapolation-on-5-long-descriptive-prompts-3x-249f-wan-2-1-1-3b-train-free-extrapolation.md>): Video Extrapolation; 5 long descriptive prompts (3x / 249f) Wan 2.1 1.3B (train-free extrapolation); VQeval: 60.4.

- [Video Extrapolation on 5 long descriptive prompts 4x / 333f train-free extrapolation Wan 2.1 1.3B](<https://www.sota2.com/research/sota/video-extrapolation-on-5-long-descriptive-prompts-4x-333f-train-free-extrapolation-wan-2-1-1-3b.md>): Video Extrapolation; 5 long descriptive prompts 4x / 333f train-free extrapolation Wan 2.1 1.3B; VQEval Score: 58.8.

- [Video Generation on VBench HunyuanVideo](<https://www.sota2.com/research/sota/video-generation-on-vbench-hunyuanvideo.md>): Video Generation; VBench HunyuanVideo; Consistency: 94.68.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2511.20123>)

[PDF](<https://arxiv.org/pdf/2511.20123.pdf>)

[Implementation](<https://github.com/thu-ml/DiT-Extrapolation>)
