# TempAct: Advancing Temporal Plausibility in Autoregressive Video Generation via Planner-Executor RL

Source: [TempAct: Advancing Temporal Plausibility in Autoregressive Video Generation via Planner-Executor RL](<https://www.sota2.com/research/paper/6a41fbf3206542251d7cca47>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-07-04T05:01:04.041Z

Authors: Jing Wang, Xiangxin Zhou, Jiajun Liang, Kaiqi Liu, Wanyuan Pang, Zhenyu Xie, Tianyu Pang, Xiaodan Liang

Published: 2026-06-26T12:19:28.000Z

## Abstract

Autoregressive (AR) video diffusion models enable low-latency streaming generation by synthesizing videos chunk by chunk with cached visual context, but this chunk-wise formulation makes temporal instruction following ambiguous. A single global prompt does not specify which sub-event should be realized in each chunk, while naively switching to step-wise prompts often leads to delayed reactions, blended step semantics, and error propagation across prompt transitions. These failures are difficult to address with supervised fine-tuning or distillation alone: SFT suffers from exposure bias, while rollout-based distillation still optimizes low-level denoising or teacher-distribution matching rather than directly enforcing action ordering and prompt-transition correctness. We address these challenges with TempAct, a planner--executor reinforcement learning framework that jointly optimizes temporal decomposition and step-conditioned execution for temporally plausible AR video generation. TempAct uses an LLM planner to explore span-aware step prompts that are executable by the video model, and trains an AR diffusion executor to follow these prompts under its own generated histories. Its key mechanism is hierarchical group exploration: candidate plans form planning groups, and each plan induces an execution group of multiple continuations from a shared visual context, enabling plan-level credit assignment for long-horizon temporal outcomes and executor-level credit assignment for prompt-switch behavior. We further design hierarchical rewards that combine plan-quality and full-video temporal feedback for the planner with local transition-level step-following rewards, aesthetic regularization, and KL constraints for the executor. Experiments on Self-Forcing and LongLive show that TempAct improves temporal consistency while preserving overall visual quality.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Video Generation on Temporal Order Simple Set, 1-2 Steps](<https://www.sota2.com/research/sota/video-generation-on-temporal-order-simple-set-1-2-steps.md>): Video Generation; Temporal Order Simple Set, 1-2 Steps; Temporal Following Score (Qwen): 50.8.

- [Video Generation on Temporal Order Hard Set, 3-4 Steps](<https://www.sota2.com/research/sota/video-generation-on-temporal-order-hard-set-3-4-steps.md>): Video Generation; Temporal Order Hard Set, 3-4 Steps; Temporal Following Score (Qwen Judge): 0.352.

- [Video Generation on Temporal Order Full Average](<https://www.sota2.com/research/sota/video-generation-on-temporal-order-full-average.md>): Video Generation; Temporal Order Full Average; Average Temporal-Following Score: 48.8.

- [Text-to-Video Generation on PickScore](<https://www.sota2.com/research/sota/text-to-video-generation-on-pickscore.md>): Text-to-Video Generation; PickScore; PickScore: 20.8.

- [Video Generation on VBench (Quality, Semantic, Total)](<https://www.sota2.com/research/sota/video-generation-on-vbench-quality-semantic-total.md>): Video Generation; VBench; Quality Score: 82.71.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2606.28016>)

[PDF](<https://arxiv.org/pdf/2606.28016.pdf>)