# FlowPortrait: Reinforcement Learning for Audio-Driven Portrait Video Generation

Source: [FlowPortrait: Reinforcement Learning for Audio-Driven Portrait Video Generation](<https://www.sota2.com/research/paper/69a66bde2d603e6de3e3ccd6>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-04T05:00:05.078Z

Authors: Weiting Tan, Andy T. Liu, Ming Tu, Xinghua Qu, Philipp Koehn, Lu Lu

Published: 2026-02-25T22:08:15.000Z

## Abstract

Generating realistic talking-head videos remains challenging due to persistent issues such as imperfect lip synchronization, unnatural motion, and evaluation metrics that correlate poorly with human perception. We propose FlowPortrait, a reinforcement-learning framework for audio-driven portrait animation built on a multimodal backbone for autoregressive audio-to-video generation. FlowPortrait introduces a human-aligned evaluation system based on Multimodal Large Language Models (MLLMs) to assess lip-sync accuracy, expressiveness, and motion quality. These signals are combined with perceptual and temporal consistency regularizers to form a stable composite reward, which is used to post-train the generator via Group Relative Policy Optimization (GRPO). Extensive experiments, including both automatic evaluations and human preference studies, demonstrate that FlowPortrait consistently produces higher-quality talking-head videos, highlighting the effectiveness of reinforcement learning for portrait animation.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Talking-head generation on Internal talking-head dataset In-domain (test)](<https://www.sota2.com/research/sota/talking-head-generation-on-internal-talking-head-dataset-in-domain-test.md>): Talking Head Generation; Internal talking-head dataset In-domain (test); Lip-sync Score: 4.42.

- [Talking-head generation on Internal talking-head dataset Out-domain (test)](<https://www.sota2.com/research/sota/talking-head-generation-on-internal-talking-head-dataset-out-domain-test.md>): Talking Head Generation; Internal talking-head dataset Out-domain (test); Lip Sync Score: 4.5.

- [Portrait Animation on Human evaluation (test)](<https://www.sota2.com/research/sota/portrait-animation-on-human-evaluation-test.md>): Portrait Animation; Human evaluation (test); Lip-sync Score: 4.16.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2603.00159>)

[PDF](<https://arxiv.org/pdf/2603.00159.pdf>)
