Predictive LLM Routing on HELM Lite
64.3OpenAI PerformanceOracle
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Oracle2025.05 | 64.3 | 63.75 | 66.74 | |
| Linear2025.05 | 55.61 | 53.54 | 52.64 | |
| MLPRepresentation=Matrix Factorization (MF)2025.05 | 55.6 | 53.21 | 51.46 | |
| LinearRepresentation=Matrix Factorization (MF)2025.05 | 55.31 | 53.4 | 51.93 | |
| kNNk=1002025.05 | 54.73 | 53.4 | 52.18 | |
| MLP2025.05 | 54.43 | 50.82 | 51.51 | |
| kNNk=102025.05 | 53.93 | 52.66 | 50.92 | |
| Graphk=102025.05 | 53.34 | 45.19 | 50.45 | |
| D-Attnk=102025.05 | 53.31 | 50.17 | 51.38 | |
| Attnk=102025.05 | 53.07 | 52.35 | 51.89 | |
| Graphk=1002025.05 | 52.7 | 51.89 | 51.41 | |
| Attnk=1002025.05 | 52.23 | 52.12 | 50.61 | |
| D-Attnk=1002025.05 | 51.49 | 49.43 | 51.82 | |
| Graphk=1002025.05 | 50.47 | 44.62 | 50.72 | |
| kNNk=102025.05 | 49.75 | 44.85 | 49.48 | |
| MLP2025.05 | 49.3 | 45.37 | 48.95 | |
| D-Attnk=102025.05 | 49.29 | 42.06 | 48.33 | |
| Attnk=1002025.05 | 49.13 | 44.31 | 49.17 | |
| D-Attnk=1002025.05 | 48.95 | 40.76 | 48.6 | |
| Random2025.05 | 48.94 | 41.89 | 45.15 | |
| kNNk=1002025.05 | 48.93 | 41.86 | 48.63 | |
| Graphk=102025.05 | 48.84 | 43.11 | 48.55 | |
| MLPMatrix Factorization=true2025.05 | 48.6 | 43.55 | 48.72 | |
| LinearMatrix Factorization=true2025.05 | 48.59 | 42.57 | 48.73 | |
| Linear2025.05 | 48.53 | 42.13 | 48.69 | |
| Attnk=102025.05 | 48.45 | 42.08 | 48.04 |