Question Answering on MMLU Pro (Accuracy and Cost Evaluation)
84AccuracyQwen3-235B
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Qwen3-235BCategory=Heuristic2026.06 | 84 | 3.749 | |
| ENTROROUTER2026.06 | 80.6 | 2.3 | |
| Qwen3-30BCategory=Heuristic2026.06 | 80.3 | 1.874 | |
| RandomCategory=Heuristic2026.06 | 79.6 | 1.927 | |
| FORCCategory=Existing Routing Architectures2026.06 | 78.5 | 1.644 | |
| RouteLLMCategory=Existing Routing Architectures2026.06 | 77.6 | 0.947 | |
| Qwen3-4BCategory=Heuristic2026.06 | 75 | 0.168 | |
| MACABackbone=Llama-3.1-70B2026.05 | 64.57 | 2,438.6 | |
| DyLANBackbone=Llama-3.1-70B2026.05 | 60.09 | 12,068.9 | |
| MaASBackbone=Llama-3.1-70B2026.05 | 58.74 | 2,868.2 | |
| AgentPruneBackbone=Llama-3.1-70B2026.05 | 57.74 | 3,128.7 | |
| CoTBackbone=Llama-3.1-70B2026.05 | 54.25 | 274.1 | |
| VanillaBackbone=Llama-3.1-70B2026.05 | 53.5 | 250.6 | |
| MACA (Ours)Model=Llama-3.1-8B, Structure Adaptation=✓, Orchestration Adaptation=✓2026.05 | 52.67 | 2,117.8 | |
| xRouterCategory=Existing Routing Architectures2026.06 | 52.1 | 0.003 | |
| PuppeteerModel=Llama-3.1-8B, Structure Adaptation=✗, Orchestration Adaptation=✓2026.05 | 51.25 | 4,097.8 | |
| Router-R1Category=Existing Routing Architectures2026.06 | 49.6 | 0.002 | |
| DyLANModel=Llama-3.1-8B, Structure Adaptation=✗, Orchestration Adaptation=✗2026.05 | 47.26 | 11,874.6 | |
| MaASModel=Llama-3.1-8B, Structure Adaptation=✓, Orchestration Adaptation=✗2026.05 | 43.94 | 2,978.4 | |
| AgentPruneModel=Llama-3.1-8B, Structure Adaptation=✓, Orchestration Adaptation=✗2026.05 | 43.53 | 2,713.6 | |
| AgentVerseModel=Llama-3.1-8B, Structure Adaptation=✗, Orchestration Adaptation=✗2026.05 | 41.69 | 6,193.2 | |
| SC (CoT×5)Model=Llama-3.1-8B, Structure Adaptation=✗, Orchestration Adaptation=✗2026.05 | 41.62 | 2,406.9 | |
| CoTModel=Llama-3.1-8B, Structure Adaptation=✗, Orchestration Adaptation=✗2026.05 | 39.24 | 379.5 | |
| ComplexCoTModel=Llama-3.1-8B, Structure Adaptation=✗, Orchestration Adaptation=✗2026.05 | 38.54 | 1,304.2 | |
| VanillaModel=Llama-3.1-8B, Structure Adaptation=✗, Orchestration Adaptation=✗2026.05 | 38.5 | 246.7 | |
| Self-RefineModel=Llama-3.1-8B, Structure Adaptation=✗, Orchestration Adaptation=✗2026.05 | 33.75 | 1,646.5 | |
| MacNetModel=Llama-3.1-8B, Structure Adaptation=✗, Orchestration Adaptation=✗2026.05 | 25.47 | 10,284.4 |