LLM Routing on MMLU Mathematics and History Error Only setting
99.67B (Str) Deferral RatioTDEF
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| TDEFSetting=Error Only (beta = 0), Expert Models=Qwen 2.5 (7B, 1.5B, 0.5B), Router Model=DeBERTa-v3-xsmall, Fine-tuning epochs=5, Learning rate=5x10−5, Optimizer=AdamW2026.04 | 99.6 | 0.438 | 0.4 | 0 | |
| MILDSetting=Error Only (beta = 0), Expert Models=Qwen 2.5 (7B, 1.5B, 0.5B), Router Model=DeBERTa-v3-xsmall, Fine-tuning epochs=5, Learning rate=5x10−5, Optimizer=AdamW2026.04 | 83.3 | 0.425 | 10.4 | 6.2 | |
| OptimalSetting=Error Only (beta = 0), Expert Models=Qwen 2.5 (7B, 1.5B, 0.5B)2026.04 | 82.2 | — | 12.8 | 5 |