Code Generation on APPS+ (Pass@k across splits)
1.94Pass@1 (Introductory)SFT
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| SFTBackbone=Qwen-2.5-Coder 0.5B, Training=Supervised Fine-Tuning2026.05 | 1.94 | 1.87 | 0.16 | 5.9 | 8.1 | 0.87 | |
| RLOO with exponential advantageBackbone=Qwen-2.5-Coder 0.5B, Training=Offline RL with exponential advantage2026.05 | 1.67 | 2.15 | 0.1 | 5.4 | 8.6 | 0.7 | |
| RLOO with GRPO advantageBackbone=Qwen-2.5-Coder 0.5B, Training=Offline RL with GRPO advantage2026.05 | 0.07 | 0.59 | 0.05 | 0.4 | 3.5 | 0.35 | |
| RLOOBackbone=Qwen-2.5-Coder 0.5B, Training=Reinforce Leave-One-Out2026.05 | 0.06 | 0.12 | 0.05 | 0.5 | 1.1 | 0.35 | |
| BASEBackbone=Qwen-2.5-Coder 0.5B, Training=Base2026.05 | 0.03 | 0.06 | 0 | 0.3 | 0.6 | 0 |