LLM Routing on MMLU Mathematics and History Error + Cost setting
31.2Deferral Ratio (7B, Str)Optimal
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| OptimalSetting=Error + Cost (beta = [1, .6, .1]), Expert Models=Qwen 2.5 (7B, 1.5B, 0.5B)2026.04 | 31.2 | — | 12 | 56.8 | |
| MILDSetting=Error + Cost (beta = [1, .6, .1]), Expert Models=Qwen 2.5 (7B, 1.5B, 0.5B), Router Model=DeBERTa-v3-xsmall, Fine-tuning epochs=5, Learning rate=5x10−5, Optimizer=AdamW2026.04 | 17.9 | 0.813 | 8.3 | 73.8 | |
| TDEFSetting=Error + Cost (beta = [1, .6, .1]), Expert Models=Qwen 2.5 (7B, 1.5B, 0.5B), Router Model=DeBERTa-v3-xsmall, Fine-tuning epochs=5, Learning rate=5x10−5, Optimizer=AdamW2026.04 | 0 | 0.928 | 0 | 100 |