Code Generation on MBPP Guided
77.19Pass Rateo4-mini
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| o4-miniProcessing=Postprocessed2026.01 | 77.19 | 22.63 | 0.18 | |
| Claude 3.7 SonnetProcessing=Postprocessed2026.01 | 74.74 | 24.91 | 0.35 | |
| GPT-4.1-MiniProcessing=Postprocessed2026.01 | 71.93 | 22.46 | 5.61 | |
| Gemini-2.0-FlashProcessing=Postprocessed2026.01 | 71.93 | 27.02 | 1.05 | |
| Qwen2.5-7b-InstructProcessing=Postprocessed2026.01 | 54.04 | 44.39 | 1.58 | |
| Qwen2.5-3b-CoderProcessing=Postprocessed2026.01 | 40.53 | 49.3 | 10.18 |