Code Generation on MBPP (tau, Speed, Score)
91.39Overall ScoreCoSpec
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| CoSpecSetting=T = 0, Target=LLaMA-70B2026.05 | 91.39 | 3.92 | 20.96 | |
| CoSpecSetting=T = 1, Target=LLaMA-70B2026.05 | 90.11 | 3.97 | 21.13 | |
| AASDSetting=T = 0, Target=LLaMA-70B2026.05 | 90.03 | 1.72 | 2.08 | |
| AASDSetting=T = 1, Target=LLaMA-70B2026.05 | 89.4 | 1.75 | 2.16 | |
| Target onlySetting=T = 0, Target=LLaMA-70B2026.05 | 89.18 | 1 | 1 | |
| SpSSetting=T = 0, Target=LLaMA-70B2026.05 | 89.18 | 1.68 | 10.43 | |
| RESTSetting=T = 0, Target=LLaMA-70B2026.05 | 89.18 | 2.01 | 2.34 | |
| EAGLE-3Setting=T = 0, Target=LLaMA-70B2026.05 | 89.18 | 3.77 | 5.45 | |
| Target onlySetting=T = 1, Target=LLaMA-70B2026.05 | 88.73 | 1 | 1 | |
| SpSSetting=T = 1, Target=LLaMA-70B2026.05 | 88.73 | 1.51 | 6.08 | |
| RESTSetting=T = 1, Target=LLaMA-70B2026.05 | 88.73 | 1.94 | 2.24 | |
| EAGLE-3Setting=T = 1, Target=LLaMA-70B2026.05 | 88.73 | 3.58 | 5.15 | |
| FLySetting=T = 0, Target=LLaMA-70B2026.05 | 88.62 | 2.45 | 11.76 | |
| FLySetting=T = 1, Target=LLaMA-70B2026.05 | 88.42 | 2.69 | 12.33 | |
| ScoreFlow2026.06 | 84.7 | — | — | |
| AFlow2026.06 | 82.9 | — | — | |
| CoSpecSetting=T = 1, Target=Qwen3-32B2026.05 | 80.09 | 1.96 | 17.97 | |
| CoSpecSetting=T = 0, Target=Qwen3-32B2026.05 | 79.1 | 1.97 | 18.01 | |
| MetaFlowgeneration_mode=single-inference generation2026.06 | 77.5 | — | — | |
| Target onlySetting=T = 1, Target=Qwen3-32B2026.05 | 76.11 | 1 | 1 | |
| SpSSetting=T = 1, Target=Qwen3-32B2026.05 | 76.11 | 1.27 | 9.41 | |
| EAGLE-3Setting=T = 1, Target=Qwen3-32B2026.05 | 76.11 | 1.83 | 3.8 | |
| Target onlySetting=T = 0, Target=Qwen3-32B2026.05 | 75.92 | 1 | 1 | |
| SpSSetting=T = 0, Target=Qwen3-32B2026.05 | 75.92 | 1.33 | 9.52 | |
| EAGLE-3Setting=T = 0, Target=Qwen3-32B2026.05 | 75.92 | 1.91 | 3.89 | |
| AASDSetting=T = 1, Target=Qwen3-32B2026.05 | 75.72 | 1.58 | 2 | |
| AASDSetting=T = 0, Target=Qwen3-32B2026.05 | 75.6 | 1.59 | 2.03 | |
| FLySetting=T = 0, Target=Qwen3-32B2026.05 | 75.42 | 1.58 | 10.55 | |
| FLySetting=T = 1, Target=Qwen3-32B2026.05 | 75.38 | 1.58 | 10.12 | |
| CoT SC2026.06 | 71.3 | — | — | |
| CoT2026.06 | 70.4 | — | — | |
| MultiPersona2026.06 | 70.4 | — | — | |
| Self Refine2026.06 | 70 | — | — | |
| IO2026.06 | 69.5 | — | — | |
| MedPrompt2026.06 | 69.2 | — | — | |
| ADAS2026.06 | 68.7 | — | — |