Question Answering on Natural Questions (LM, Acc., Time)
63.41LM ScoreGoldRouter
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| GoldRouterModel=Qwen-Plus2025.01 | 63.41 | 83.02 | 12.97 | |
| RIRSModel=Qwen-Plus2025.01 | 62.05 | 80.05 | 23.3 | |
| RIRSModel=Mixed, Note=RAG agents use LLaMA-3.1-8B, other modules use Qwen-Plus2025.01 | 55.98 | 73.32 | 20.98 | |
| ChameleonModel=Qwen-Plus2025.01 | 55.48 | 73.92 | 24.9 | |
| GoldRouterModel=LLaMA-3.1-8B2025.01 | 55.13 | 69.7 | 6.29 | |
| RouterDCModel=Qwen-Plus2025.01 | 53.93 | 71.09 | 24.91 | |
| RIRSModel=LLaMA-3.1-8B2025.01 | 50.98 | 63.96 | 10.51 | |
| ChameleonModel=LLaMA-3.1-8B2025.01 | 49.68 | 62.73 | 15.27 | |
| RouterDCModel=LLaMA-3.1-8B2025.01 | 47.26 | 60.43 | 13.11 |