# Optimizing Visual Generative Models via Distribution-wise Rewards Source: [Optimizing Visual Generative Models via Distribution-wise Rewards]() Collection index: [llms.txt]() Data updated: 2026-07-06T06:00:00.783Z Authors: Ruihang Li, Mengde Xu, Shuyang Gu, Leigang Qu, Fuli Feng, Han Hu, Wenjie Wang Published: 2026-07-02T15:08:56.000Z ## Abstract Conventional reinforcement learning strategies for visual generation typically employ sample-wise reward functions, yet this practice frequently results in reward hacking that degrades image diversity and introduces visual anomalies. To address these limitations, we present a novel framework that finetunes generative models using distribution-wise rewards, ensuring better alignment with real-world data distributions. Unlike rewards that evaluate samples individually, distribution-wise reward accounts for the data distribution of the samples, mitigating the mode collapse problem that occurs when all samples optimize towards the same direction independently. To overcome the prohibitive computational cost of estimating these rewards, we introduce a subset-replace strategy that efficiently provides reward signals by updating only a small subset of a generated reference set. Additionally, we apply RL to optimize post-hoc model merging coefficients, potentially mitigating the train-inference inconsistency caused by introducing stochastic differential equation (SDE) in regular RL practices. Extensive experiments show our approach significantly improves FID-50K across various base models, from 8.30 to 5.77 for SiT and from 3.74 to 3.52 for EDM2. Qualitative evaluation also confirms that our method enhances perceptual quality while preserving sample diversity. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Image Generation on ImageNet 512x512 (test val 50K images)](): Image Generation; ImageNet 512x512 (test val 50K images); FID: 2.52. - [Image Generation on ImageNet 256x256 (val)](): Image Generation; ImageNet 256x256 (val); FID: 5.77. ## Original sources and implementation [Original paper]() [PDF]()