Automated Program Repair on QuixBugs-Java 40 bugs
100Pass@1 RateChatRepair
Evaluation results
| Method | Links | |
|---|---|---|
| ChatRepairBackbone=GPT-3.5-turbo2026.05 | 100 | |
| + Stage III (PPO, Rseq only)Params=32B2026.05 | 95 | |
| BOOSTAPR (+ Rline)Params=32B2026.05 | 95 | |
| Lingma SWE-GPTBackbone=Qwen2.5-72B, Params=72B2026.05 | 92.5 | |
| SWE-FixerBackbone=Qwen2.5-72B, Params=72B2026.05 | 92.5 | |
| + Stage I (SFT)Params=32B2026.05 | 92.5 | |
| SWE-GymBackbone=Qwen2.5-Coder-32B, Params=32B2026.05 | 90 | |
| SWE-RLBackbone=Llama-3-70B, Params=70B2026.05 | 90 | |
| Qwen2.5-Coder-32B (base)Params=32B2026.05 | 90 | |
| AgentlessBackbone=GPT-4o2026.05 | 87.5 | |
| SWE-agentBackbone=Claude 3.5 Sonnet2026.05 | 85 | |
| AutoCodeRoverBackbone=GPT-4o2026.05 | 82.5 | |
| RLEFBackbone=Llama-3-8B, Params=8B2026.05 | 80 | |
| RepairLLaMABackbone=CodeLlama-7B, Params=7B2026.05 | 75 | |
| CodeRLBackbone=CodeT5-large, Params=770M2026.05 | 67.5 | |
| KNODBackbone=CodeT5-base, Params=220M2026.05 | 62.5 |