LLM Routing on BBEH
66.4Top-1 AccuracyORACLE
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ORACLEModel Pool=Pool Large, Training Data Source=Gemini-2.5-Flash2026.01 | 66.4 | — | |
| ORACLEModel Pool=Pool Small, Training Data Source=Gemini-2.5-Flash2026.01 | 56.7 | — | |
| LLMRANKModel Pool=Pool Large, Router Category=Query-answer Routers, Training Data Source=Gemini-2.5-Flash2026.01 | 34.5 | — | |
| TOP-1 / TOP-3Model Pool=Pool Large, Training Data Source=Gemini-2.5-Flash2026.01 | 34 | 34.3 | |
| CASCAL-GTModel Pool=Pool Large, Router Category=Query-answer Routers, Training Data Source=Gemini-2.5-Flash2026.01 | 33.1 | 32.6 | |
| SMOOTHIEModel Pool=Pool Large, Router Category=Query-only Routers, Training Data Source=Gemini-2.5-Flash2026.01 | 32.1 | 32 | |
| AVENGERSModel Pool=Pool Large, Router Category=Query-answer Routers, Training Data Source=Gemini-2.5-Flash2026.01 | 32 | — | |
| CASCALModel Pool=Pool Large, Router Category=Query-only Routers, Training Data Source=Gemini-2.5-Flash2026.01 | 30.5 | 33.9 | |
| LLMRANKModel Pool=Pool Small, Router Category=Query-answer Routers, Training Data Source=Gemini-2.5-Flash2026.01 | 27.3 | — | |
| TOP-1 / TOP-3Model Pool=Pool Small, Training Data Source=Gemini-2.5-Flash2026.01 | 25.1 | 25 | |
| CASCAL-GTModel Pool=Pool Small, Router Category=Query-answer Routers, Training Data Source=Gemini-2.5-Flash2026.01 | 25 | 24.3 | |
| AVENGERSModel Pool=Pool Small, Router Category=Query-answer Routers, Training Data Source=Gemini-2.5-Flash2026.01 | 24.3 | — | |
| CASCALModel Pool=Pool Small, Router Category=Query-only Routers, Training Data Source=Gemini-2.5-Flash2026.01 | 23.8 | 24.1 | |
| SMOOTHIEModel Pool=Pool Small, Router Category=Query-only Routers, Training Data Source=Gemini-2.5-Flash2026.01 | 22.1 | 23.1 |