Reasoning on ARC Challenge (Accuracy, Cost, Latency)
96.8AccuracyOracle
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| OracleType=Single LLM with Routing2026.01 | 96.8 | — | — | |
| Gemma-2-9B-itType=Single LLM2026.01 | 90.2 | — | — | |
| SMoAType=Multi-LLMs2026.01 | 89.4 | 0.47 | 10.45 | |
| RouteMoAType=Ours2026.01 | 88.2 | 0.28 | 9.73 | |
| RouterDCType=Single LLM with Routing2026.01 | 87.3 | — | — | |
| MoAType=Multi-LLMs2026.01 | 87 | 2.27 | 12.05 | |
| Qwen2.5-Coder-7B-InstructType=Single LLM2026.01 | 85.5 | — | — | |
| Ministral-8B-Instruct-2410Type=Single LLM2026.01 | 85.3 | — | — | |
| RouteLLMType=Single LLM with Routing2026.01 | 84.8 | — | — | |
| Bio-Medical-Llama-3-8BType=Single LLM2026.01 | 75.1 | — | — | |
| Qwen2.5-Math-7B-InstructType=Single LLM2026.01 | 50.6 | — | — |