# Dropout-GRPO: Variational Stochasticity for Continuous Latent Reasoning Source: [Dropout-GRPO: Variational Stochasticity for Continuous Latent Reasoning]() Collection index: [llms.txt]() Data updated: 2026-06-11T05:00:05.512Z Authors: Wooil Jung Published: 2026-06-08T21:21:42.000Z ## Abstract Group Relative Policy Optimization (GRPO) relies on the diversity of $K$ rollouts within each group; otherwise, the group-mean advantage $A^{(k)} = r^{(k)} - \\mu\_r$ collapses to zero. This presents a structural challenge for latent-reasoning models like Coconut, which feed continuous hidden states recurrently in place of discrete chain-of-thought tokens. Because the latent phase is inherently deterministic given the parameters and prompt, multiple rollouts produce identical trajectories, stalling GRPO's progress. Consequently, applying group-relative reinforcement learning to continuous latent reasoning has proven difficult. To address this, we propose sourcing the necessary stochasticity through structured dropout. By applying a single Bernoulli mask held constant across all latent recurrence steps for a given rollout, we generate essential trajectory variance. This shared mask effectively treats each rollout as a posterior sample from a variational distribution over parameters, allowing GRPO to optimize the expected reward of a Bayesian model-average policy. We provide both theoretical justification for this method -- including unbiasedness, variance reduction, and the well-definedness of the latent gradient -- and empirical validation. On GSM8K, dropout-GRPO improves a Coconut baseline from $27.29\\%$ to $29.01\\%$ pass@1, demonstrating the viability of GRPO learning for latent-reasoning models. Our work positions this as a practical, theoretically grounded approach for post-training latent-reasoning LLMs. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Mathematical Reasoning on MultiArith (pass@1)](): Mathematical Reasoning; MultiArith; Pass@1 Rate: 64.67. - [Mathematical Reasoning on SVAMP (Pass@1 Accuracy)](): Mathematical Reasoning; SVAMP; Pass@1 Accuracy: 44. - [Mathematical Reasoning on GSM8K (Acc, H100 TPS, TPF)](): Mathematical Reasoning; GSM8K; GSM8K Accuracy (%): not recorded. ## Original sources and implementation [Original paper]() [PDF]()