LLM response quality prediction on ID Claude 3.5 Haiku 20241022 (test)
0.45RMSEMIRT-Router
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| MIRT-Routeralpha=0.8, Target LLM=Claude 3.5 Haiku, scenario=ID2025.06 | 0.45 | 0.43 | 62 | 67 | |
| NIRT-Routeralpha=0.8, Target LLM=Claude 3.5 Haiku, scenario=ID2025.06 | 0.45 | 0.42 | 62 | 68 | |
| RouterBenchalpha=0.8, Target LLM=Claude 3.5 Haiku, scenario=ID2025.06 | 0.62 | 0.55 | 50 | 34 |