Automated Program Repair on HumanEval Java (164 tasks)
84.5Pass@1 RateBOOSTAPR (+ Rline)
Evaluation results
| Method | Links | |
|---|---|---|
| BOOSTAPR (+ Rline)Params=32B2026.05 | 84.5 | |
| + Stage III (PPO, Rseq only)Params=32B2026.05 | 79.4 | |
| SWE-RLBackbone=Llama-3-70B, Params=70B2026.05 | 76.2 | |
| RLEFBackbone=Llama-3-8B, Params=8B2026.05 | 74.3 | |
| SWE-FixerBackbone=Qwen2.5-72B, Params=72B2026.05 | 73.8 | |
| + Stage I (SFT)Params=32B2026.05 | 73.1 | |
| Lingma SWE-GPTBackbone=Qwen2.5-72B, Params=72B2026.05 | 72.6 | |
| ChatRepairBackbone=GPT-3.5-turbo2026.05 | 72 | |
| AgentlessBackbone=GPT-4o2026.05 | 71.3 | |
| SWE-GymBackbone=Qwen2.5-Coder-32B, Params=32B2026.05 | 70.7 | |
| SWE-agentBackbone=Claude 3.5 Sonnet2026.05 | 68.9 | |
| RepairLLaMABackbone=CodeLlama-7B, Params=7B2026.05 | 66.5 | |
| AutoCodeRoverBackbone=GPT-4o2026.05 | 65.2 | |
| Qwen2.5-Coder-32B (base)Params=32B2026.05 | 64 | |
| CodeRLBackbone=CodeT5-large, Params=770M2026.05 | 63 | |
| KNODBackbone=CodeT5-base, Params=220M2026.05 | 58.5 |