LLM Routing on Agentic Evaluation (test)
89.65AccuracyOracle
Evaluation results
| Method | Links | ||
|---|---|---|---|
| OracleType=Upper Bound, Model Basket=Earlier2026.05 | 89.65 | 12.7 | |
| GPT-4.1Type=Single LLM, Model Basket=Earlier2026.05 | 83.22 | 73.6 | |
| DistilBERT (ours)Type=Agent Router, Model Basket=Earlier2026.05 | 82.73 | 8.7 | |
| GPT-4.1-miniType=Single LLM, Model Basket=Earlier2026.05 | 82.18 | 10.5 | |
| GPT-5Type=Single LLM, Model Basket=Earlier2026.05 | 79.18 | 49.9 | |
| LengthType=Heuristic Router, Model Basket=Earlier2026.05 | 78.95 | 12.8 | |
| GPT-5-nanoType=Single LLM, Model Basket=Earlier2026.05 | 78.8 | 1.8 | |
| Chat-fine-tunedType=Chat Router, Model Basket=Earlier2026.05 | 77.47 | 7.7 | |
| GPT-5-miniType=Single LLM, Model Basket=Earlier2026.05 | 77.24 | 8 | |
| GPT-4.1-nanoType=Single LLM, Model Basket=Earlier2026.05 | 76.67 | 3.2 | |
| Num. Tool CallsType=Heuristic Router, Model Basket=Earlier2026.05 | 67.84 | 66.9 | |
| GPT-5-chatType=Single LLM, Model Basket=Earlier2026.05 | 62.76 | 22.7 | |
| Num. TurnsType=Heuristic Router, Model Basket=Earlier2026.05 | 62.67 | 54 | |
| o4-miniType=Single LLM, Model Basket=Earlier2026.05 | 60.93 | 23.6 |