Code Generation on MBPP Hard-Guided
72.44Pass Rateo4-mini
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| o4-miniProcessing=Postprocessed2026.01 | 72.44 | 26.28 | 1.28 | |
| GPT-4.1-MiniProcessing=Postprocessed2026.01 | 68.38 | 27.78 | 3.85 | |
| Claude 3.7 SonnetProcessing=Postprocessed2026.01 | 66.03 | 33.33 | 0.64 | |
| Gemini-2.0-FlashProcessing=Postprocessed2026.01 | 62.39 | 35.04 | 2.56 | |
| Qwen2.5-7b-InstructProcessing=Postprocessed2026.01 | 39.1 | 56.84 | 4.06 | |
| Qwen2.5-3b-CoderProcessing=Postprocessed2026.01 | 29.06 | 58.12 | 12.82 |