Question Answering on HotpotQA (LM, F1, Acc., Rounds, Time)
58.17LM ScoreGoldRouter
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| GoldRouterModel=Qwen-Plus2025.01 | 58.17 | 63.99 | 81.01 | 1.22 | 22.85 | |
| RIRSModel=Qwen-Plus2025.01 | 54.01 | 59.62 | 75.63 | 1.44 | 34.71 | |
| ChameleonModel=Qwen-Plus2025.01 | 52.12 | 57.55 | 73.19 | 1.43 | 39.05 | |
| RIRSModel=Mixed, Note=RAG agents use LLaMA-3.1-8B, other modules use Qwen-Plus2025.01 | 51.68 | 56.65 | 69.33 | 1.83 | 41.04 | |
| RouterDCModel=Qwen-Plus2025.01 | 51.34 | 56.84 | 67.24 | 1.46 | 44.12 | |
| GoldRouterModel=LLaMA-3.1-8B2025.01 | 50.6 | 56.26 | 64.94 | 1.55 | 17.07 | |
| RIRSModel=LLaMA-3.1-8B2025.01 | 47.57 | 52.56 | 58.69 | 2.04 | 19.93 | |
| ChameleonModel=LLaMA-3.1-8B2025.01 | 41.08 | 46.59 | 50.2 | 1.85 | 24.77 | |
| RouterDCModel=LLaMA-3.1-8B2025.01 | 38.31 | 43.11 | 47.74 | 1.78 | 23.63 |