Automated Program Repair on Defects4J 835 bugs v2.0
24.8Pass@1BOOSTAPR (+ Rline)
Evaluation results
| Method | Links | |
|---|---|---|
| BOOSTAPR (+ Rline)Params=32B2026.05 | 24.8 | |
| + Stage III (PPO, Rseq only)Params=32B2026.05 | 19.2 | |
| RepairLLaMABackbone=CodeLlama-7B, Params=7B2026.05 | 17.2 | |
| SWE-RLBackbone=Llama-3-70B, Params=70B2026.05 | 16.8 | |
| SWE-FixerBackbone=Qwen2.5-72B, Params=72B2026.05 | 15.2 | |
| + Stage I (SFT)Params=32B2026.05 | 14.9 | |
| Lingma SWE-GPTBackbone=Qwen2.5-72B, Params=72B2026.05 | 14.5 | |
| ChatRepairBackbone=GPT-3.5-turbo2026.05 | 14.4 | |
| SWE-GymBackbone=Qwen2.5-Coder-32B, Params=32B2026.05 | 13.1 | |
| AgentlessBackbone=GPT-4o2026.05 | 12.4 | |
| Qwen2.5-Coder-32B (base)Params=32B2026.05 | 11.3 | |
| SWE-agentBackbone=Claude 3.5 Sonnet2026.05 | 10.8 | |
| AutoCodeRoverBackbone=GPT-4o2026.05 | 9.6 | |
| RLEFBackbone=Llama-3-8B, Params=8B2026.05 | 8.4 | |
| KNODBackbone=CodeT5-base, Params=220M2026.05 | 6 | |
| CodeRLBackbone=CodeT5-large, Params=770M2026.05 | 5.8 |