# Interactive Critique-Revision Training for Reliable Structured LLM Generation

Source: [Interactive Critique-Revision Training for Reliable Structured LLM Generation](<https://www.sota2.com/research/paper/6a02b3f8dbc836b87d82459d>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-13T05:00:05.563Z

Authors: Fei Xu Yu, Zuyuan Zhang, Mahdi Imani, Nathaniel D. Bastian, Tian Lan

Published: 2026-05-08T17:00:38.000Z

## Abstract

In structured decision-making workflows such as form filling, compliance checking, and maintenance reporting, LLM outputs must be locally correct, globally consistent, and auditable against task-specific rules. Existing refinement methods often rely on heuristic debate, self-play, or LLM-generated supervision, creating a second-order assurance problem. We propose DPA-GRPO (Dual Paired-Action Group-Relative Policy Optimization), a paired-action training method for a two-player generator--verifier game with structured verifier interventions. The generator proposes outputs and may revise them when challenged; the verifier either remains silent or raises a safety assurance case (SAC) containing a claim, argument, and evidence. These SAC/no-SAC and KEEP/REVISE decisions induce paired counterfactual action groups, which DPA-GRPO uses for role-specific KL-regularized GRPO updates. We analyze the unregularized game and show that positive probability on strictly lower-reward intervention or revision actions creates a profitable unilateral deviation. Under standard stochastic-approximation assumptions, DPA-GRPO tracks the corresponding game ODE, whose isolated asymptotically stable limit points are stationary and candidate local equilibria under role-wise local optimality. Experiments on TaxCalcBench TY24 show that DPA-GRPO improves structured decision accuracy over zero-shot generation and generator-only RL baselines across Qwen3-4B and Qwen3-8B. Training increases correct silent acceptance, reduces missed errors, and improves calibrated revision behavior, indicating gains for both generator and verifier.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Structured tax-form completion on TaxCalcBench TY24 (train)](<https://www.sota2.com/research/sota/structured-tax-form-completion-on-taxcalcbench-ty24-train.md>): Structured tax-form completion; TaxCalcBench TY24 (train); Train-A Score: 90.

- [Structured tax-form completion on TaxCalcBench TY24 (test)](<https://www.sota2.com/research/sota/structured-tax-form-completion-on-taxcalcbench-ty24-test.md>): Structured tax-form completion; TaxCalcBench TY24 (test); Case 1 Accuracy: 52.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2605.08327>)

[PDF](<https://arxiv.org/pdf/2605.08327.pdf>)