List your company
ResearchFormerly WizWand

Question Answering on MMLU Pro (Accuracy and Cost Evaluation)

84AccuracyQwen3-235B
23.128838.931954.73570.5381May 25, 2026May 30, 2026Jun 5, 2026Jun 11, 2026Jun 16, 2026Jun 22, 2026Jun 28, 2026
Updated 4mo ago

Evaluation results

MethodLinks
Qwen3-235BCategory=Heuristic2026.06843.749
ENTROROUTER2026.0680.62.3
Qwen3-30BCategory=Heuristic2026.0680.31.874
RandomCategory=Heuristic2026.0679.61.927
FORCCategory=Existing Routing Architectures2026.0678.51.644
RouteLLMCategory=Existing Routing Architectures2026.0677.60.947
Qwen3-4BCategory=Heuristic2026.06750.168
MACABackbone=Llama-3.1-70B2026.0564.572,438.6
DyLANBackbone=Llama-3.1-70B2026.0560.0912,068.9
MaASBackbone=Llama-3.1-70B2026.0558.742,868.2
AgentPruneBackbone=Llama-3.1-70B2026.0557.743,128.7
CoTBackbone=Llama-3.1-70B2026.0554.25274.1
VanillaBackbone=Llama-3.1-70B2026.0553.5250.6
MACA (Ours)Model=Llama-3.1-8B, Structure Adaptation=✓, Orchestration Adaptation=✓2026.0552.672,117.8
xRouterCategory=Existing Routing Architectures2026.0652.10.003
PuppeteerModel=Llama-3.1-8B, Structure Adaptation=✗, Orchestration Adaptation=✓2026.0551.254,097.8
Router-R1Category=Existing Routing Architectures2026.0649.60.002
DyLANModel=Llama-3.1-8B, Structure Adaptation=✗, Orchestration Adaptation=✗2026.0547.2611,874.6
MaASModel=Llama-3.1-8B, Structure Adaptation=✓, Orchestration Adaptation=✗2026.0543.942,978.4
AgentPruneModel=Llama-3.1-8B, Structure Adaptation=✓, Orchestration Adaptation=✗2026.0543.532,713.6
AgentVerseModel=Llama-3.1-8B, Structure Adaptation=✗, Orchestration Adaptation=✗2026.0541.696,193.2
SC (CoT×5)Model=Llama-3.1-8B, Structure Adaptation=✗, Orchestration Adaptation=✗2026.0541.622,406.9
CoTModel=Llama-3.1-8B, Structure Adaptation=✗, Orchestration Adaptation=✗2026.0539.24379.5
ComplexCoTModel=Llama-3.1-8B, Structure Adaptation=✗, Orchestration Adaptation=✗2026.0538.541,304.2
VanillaModel=Llama-3.1-8B, Structure Adaptation=✗, Orchestration Adaptation=✗2026.0538.5246.7
Self-RefineModel=Llama-3.1-8B, Structure Adaptation=✗, Orchestration Adaptation=✗2026.0533.751,646.5
MacNetModel=Llama-3.1-8B, Structure Adaptation=✗, Orchestration Adaptation=✗2026.0525.4710,284.4