Coding on MBPP Base (pass@1)
92Pass@1HASP-Evolve + RS
Evaluation results
| Method | Links | |
|---|---|---|
| HASP-Evolve + RSSize=7B-Inst, Training Setting=Training-based, Backbone=Qwen2.5-7B-Instruct2026.05 | 92 | |
| GPT-4oSize=~200B, Training Setting=Training-free / inference-time2026.05 | 91 | |
| HASP-Intervention (w. Teacher)Size=7B-Inst, Training Setting=Training-free / inference-time, Backbone=Qwen2.5-7B-Instruct2026.05 | 91 | |
| GPT-4o-miniSize=~8B, Training Setting=Training-free / inference-time2026.05 | 88 | |
| HASP-Intervention (PF-only)Size=7B-Inst, Training Setting=Training-free / inference-time, Backbone=Qwen2.5-7B-Instruct2026.05 | 88 | |
| P-GRPO (Code+RM)Size=7B-Inst, Training Setting=Training-based2026.05 | 87 | |
| GRPO (Code)Size=7B-Inst, Training Setting=Training-based2026.05 | 86.7 | |
| Prompt-Only SkillsSize=7B-Inst, Training Setting=Training-free / inference-time2026.05 | 86 | |
| AceCoderRMSize=7B-Inst, Training Setting=Training-based2026.05 | 83.1 | |
| KodCode-RL Step 256Size=7B-Inst, Training Setting=Training-based2026.05 | 82.8 | |
| RA-Agent (multi-loop)Size=7B-Inst, Training Setting=Training-free / inference-time2026.05 | 82 | |
| KodCode-RL Step 128Size=7B-Inst, Training Setting=Training-based2026.05 | 81 | |
| AceCoderRuleSize=7B-Inst, Training Setting=Training-based2026.05 | 80.2 | |
| Qwen2.5-7B-InstructSize=7B-Inst, Training Setting=Training-free / inference-time2026.05 | 79.4 | |
| SFT (vanilla)Size=7B-Inst, Training Setting=Training-based2026.05 | 76 |