Question Answering on HotpotQA (test) (EM Metric)
57.2EMIdeal Model Selection
Evaluation Results
| Method | Links | |
|---|---|---|
| Ideal Model SelectionCategory=Baseline2025.10 | 57.2 | |
| R2CSelectCategory=Model Selection RAR2025.10 | 50.2 | |
| RAGEnsembleCategory=Model Selection RAR2025.10 | 46 | |
| R2CSelectCategory=Model Selection RAR, Variant=w/o clustering2025.10 | 44 | |
| Search-R1Category=Retrieval-Augmented Reasoning (RAR)2025.10 | 41.4 | |
| RAGEnsembleCategory=Model Selection RAR, Variant=w/o clustering2025.10 | 40.8 | |
| ReSearchCategory=Retrieval-Augmented Reasoning (RAR)2025.10 | 38.8 | |
| INSPOBackbone=Qwen-2.5-3B, Category=Tool-integrated Methods2025.12 | 37.7 | |
| RAGEnsembleCategory=Model Selection RAR, Variant=w/o clus. & conf.2025.10 | 37.2 | |
| INSPOBackbone=Qwen-2.5-7B, Category=Tool-integrated Methods2025.12 | 36.8 | |
| LLMBlenderCategory=Model Selection RAR2025.10 | 36 | |
| Search-R1Backbone=Qwen-2.5-7B, Category=Tool-integrated Methods2025.12 | 35.6 | |
| SFTBackbone=Qwen-2.5-7B, Category=Tool-integrated Methods2025.12 | 33.1 | |
| SelfAskCategory=Retrieval-Augmented Reasoning (RAR)2025.10 | 33 | |
| LLMBlenderCategory=Model Selection RAR, Variant=w/o clustering2025.10 | 31.8 | |
| RandomCategory=Model Selection RAR2025.10 | 31.6 | |
| Search-R1Backbone=Qwen-2.5-3B, Category=Tool-integrated Methods2025.12 | 30.9 | |
| RAGBackbone=Qwen-2.5-7B, Category=Tool-integrated Methods2025.12 | 29.9 | |
| Search-o1Category=Retrieval-Augmented Reasoning (RAR)2025.10 | 29 | |
| ReActCategory=Retrieval-Augmented Reasoning (RAR)2025.10 | 27.8 | |
| IRCoTCategory=Vanilla LLM & RAG2025.10 | 27.2 | |
| LLMBlenderCategory=Model Selection RAR, Variant=w/o clus. & conf.2025.10 | 26 | |
| RAGBackbone=Qwen-2.5-3B, Category=Tool-integrated Methods2025.12 | 25.5 | |
| FLARECategory=Vanilla LLM & RAG2025.10 | 25.2 | |
| GRPOBackbone=Qwen-2.5-7B, Category=Model Inference2025.12 | 24.2 | |
| SFTBackbone=Qwen-2.5-3B, Category=Tool-integrated Methods2025.12 | 24 | |
| DRAGINCategory=Vanilla LLM & RAG2025.10 | 23.2 | |
| CoTCategory=Vanilla LLM & RAG2025.10 | 22.2 | |
| Search-o1Backbone=Qwen-2.5-3B, Category=Tool-integrated Methods2025.12 | 22.1 | |
| SFTBackbone=Qwen-2.5-7B, Category=Model Inference2025.12 | 21.7 | |
| DirectCategory=Vanilla LLM & RAG2025.10 | 20.8 | |
| GRPOBackbone=Qwen-2.5-3B, Category=Model Inference2025.12 | 20.1 | |
| Search-o1Backbone=Qwen-2.5-7B, Category=Tool-integrated Methods2025.12 | 18.7 | |
| SFTBackbone=Qwen-2.5-3B, Category=Model Inference2025.12 | 18.6 | |
| Vanilla RAGCategory=Vanilla LLM & RAG2025.10 | 18.6 | |
| DirectBackbone=Qwen-2.5-7B, Category=Model Inference2025.12 | 18.3 | |
| IRCoTBackbone=Qwen-2.5-3B, Category=Tool-integrated Methods2025.12 | 16.4 | |
| DirectBackbone=Qwen-2.5-3B, Category=Model Inference2025.12 | 14.9 | |
| IRCoTBackbone=Qwen-2.5-7B, Category=Tool-integrated Methods2025.12 | 13.3 |