Coding on BigCodeBench Full (pass@1)
54pass@1P-GRPO (Code+RM)
Evaluation results
| Method | Links | |
|---|---|---|
| P-GRPO (Code+RM)Size=7B-Inst, Training Setting=Training-based2026.05 | 54 | |
| GRPO (Code)Size=7B-Inst, Training Setting=Training-based2026.05 | 52 | |
| HASP-Evolve + RSSize=7B-Inst, Training Setting=Training-based, Backbone=Qwen2.5-7B-Instruct2026.05 | 50 | |
| HASP-Intervention (w. Teacher)Size=7B-Inst, Training Setting=Training-free / inference-time, Backbone=Qwen2.5-7B-Instruct2026.05 | 49 | |
| KodCode-RL Step 256Size=7B-Inst, Training Setting=Training-based2026.05 | 48.2 | |
| KodCode-RL Step 128Size=7B-Inst, Training Setting=Training-based2026.05 | 47 | |
| AceCoderRMSize=7B-Inst, Training Setting=Training-based2026.05 | 46.8 | |
| AceCoderRuleSize=7B-Inst, Training Setting=Training-based2026.05 | 46.8 | |
| Qwen2.5-7B-InstructSize=7B-Inst, Training Setting=Training-free / inference-time2026.05 | 45.6 | |
| HASP-Intervention (PF-only)Size=7B-Inst, Training Setting=Training-free / inference-time, Backbone=Qwen2.5-7B-Instruct2026.05 | 42 | |
| GPT-4oSize=~200B, Training Setting=Training-free / inference-time2026.05 | 41 | |
| Prompt-Only SkillsSize=7B-Inst, Training Setting=Training-free / inference-time2026.05 | 40 | |
| SFT (vanilla)Size=7B-Inst, Training Setting=Training-based2026.05 | 34 | |
| GPT-4o-miniSize=~8B, Training Setting=Training-free / inference-time2026.05 | 32 | |
| RA-Agent (multi-loop)Size=7B-Inst, Training Setting=Training-free / inference-time2026.05 | 31 |