Code Generation on MBPP (Accuracy, Cost, Latency)
86.7AccuracyOracle
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| OracleType=Single LLM with Routing2026.01 | 86.7 | — | — | |
| Qwen2.5-Coder-7B-InstructType=Single LLM2026.01 | 79.8 | — | — | |
| RouteMoAType=Ours2026.01 | 79.8 | 0.37 | 7.31 | |
| SMoAType=Multi-LLMs2026.01 | 79.4 | 0.18 | 10.3 | |
| RouteLLMType=Single LLM with Routing2026.01 | 76.3 | — | — | |
| MoAType=Multi-LLMs2026.01 | 75.5 | 0.61 | 15.52 | |
| RouterDCType=Single LLM with Routing2026.01 | 72.6 | — | — | |
| Gemma-2-9B-itType=Single LLM2026.01 | 66.2 | — | — | |
| Ministral-8B-Instruct-2410Type=Single LLM2026.01 | 63 | — | — | |
| Qwen2.5-Math-7B-InstructType=Single LLM2026.01 | 52.9 | — | — | |
| Bio-Medical-Llama-3-8BType=Single LLM2026.01 | 16 | — | — |