# MURPHY: Feedback-Aware GRPO with Retrospective Credit Assignment for Multi-Turn Code Generation

Source: [MURPHY: Feedback-Aware GRPO with Retrospective Credit Assignment for Multi-Turn Code Generation](<https://www.sota2.com/research/paper/691414712828b7cc6937e961>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-13T05:00:12.693Z

Authors: Chanakya Ekbote, Vijay Lingam, Sujay Sanghavi, Jun Huan, Behrooz Omidvar-Tehrani, Anoop Deoras, Stefano Soatto

Published: 2025-11-11T05:03:22.000Z

## Abstract

Reinforcement Learning with Verifiable Rewards (RLVR) has become a standard recipe for post-training LLMs on reasoning tasks, with Group Relative Policy Optimization (GRPO) emerging as a leading approach. However, GRPO and its variants are inherently single-turn: they optimize from terminal rewards on isolated prompt-response pairs, leaving them poorly suited to agentic settings where models must iteratively refine solutions in response to environmental feedback. We introduce MURPHY, a multi-turn extension of GRPO for self-correcting code generation. MURPHY constructs feedback-conditioned rollout trees in which failed candidate solutions are paired with executor feedback and expanded into subsequent turns, and propagates rewards backward through the tree so that later successful refinements credit earlier attempts that surfaced informative feedback. We study two propagation strategies, Max Reward (MARS) and Mean Reward (MERS), and introduce post-rollout pruning mechanisms that reduce multi-turn optimization cost. Across three code generation benchmarks (HumanEval, MBPP, LiveCodeBench-v6) and two model families (Qwen3-1.7B/4B, OLMo-2-7B), MURPHY delivers up to 6% absolute pass@1 gains over the strongest prior multi-turn execution-feedback methods. Gains are largest on the Medium/Hard subset (+4.38/+4.20 at Iter-5), where iterative self-correction matters more.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Code Generation on MBPP (pass@1, ∆3)](<https://www.sota2.com/research/sota/code-generation-on-mbpp-pass-at-1-3.md>): Code Generation; MBPP; Pass@1: 73.33.

- [Code Generation on BigCodeBench (pass@1, ∆3)](<https://www.sota2.com/research/sota/code-generation-on-bigcodebench-pass-at-1-3.md>): Code Generation; BigCodeBench; pass@1: 41.44.

- [Code Generation on HumanEval](<https://www.sota2.com/research/sota/code-generation-on-humaneval.md>): Code Generation; HumanEval; Pass@1: 95.73.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2511.07833>)

[PDF](<https://arxiv.org/pdf/2511.07833.pdf>)