Multi-hop Question Answering on Bamboogle Wiki-18
53.4Exact Match (EM)Esearch-base
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Esearch-baseModel Backbone=Qwen2.5-7b, Search Engine=Wiki-18, Model Variant=Base2025.10 | 53.4 | 65.6 | |
| Esearch-instructModel Backbone=Qwen2.5-7b, Search Engine=Wiki-18, Model Variant=Instruct2025.10 | 45.8 | 61.2 | |
| Esearch-instructModel Backbone=Qwen2.5-3b, Search Engine=Wiki-18, Model Variant=Instruct2025.10 | 44.6 | 58.7 | |
| Esearch-baseModel Backbone=Qwen2.5-3b, Search Engine=Wiki-18, Model Variant=Base2025.10 | 41.4 | 52.9 | |
| Search-o1Model Backbone=Qwen2.5-3b, Search Engine=Wiki-182025.10 | 33.6 | 39.7 | |
| RAGModel Backbone=Qwen2.5-3b, Search Engine=Wiki-182025.10 | 30.3 | 36.4 | |
| Search-o1Model Backbone=Qwen2.5-7b, Search Engine=Wiki-182025.10 | 28.1 | 36.6 | |
| R1-baseModel Backbone=Qwen2.5-7b, Search Engine=Wiki-18, Model Variant=Base2025.10 | 27.7 | 33.5 | |
| RAGModel Backbone=Qwen2.5-7b, Search Engine=Wiki-182025.10 | 27.3 | 36.1 | |
| R1-instructModel Backbone=Qwen2.5-7b, Search Engine=Wiki-18, Model Variant=Instruct2025.10 | 26.6 | 32.9 | |
| R1-baseModel Backbone=Qwen2.5-3b, Search Engine=Wiki-18, Model Variant=Base2025.10 | 24.6 | 30.3 | |
| IRCoTModel Backbone=Qwen2.5-7b, Search Engine=Wiki-182025.10 | 23.7 | 29.6 | |
| IRCoTModel Backbone=Qwen2.5-3b, Search Engine=Wiki-182025.10 | 22.1 | 30.5 | |
| CoTModel Backbone=Qwen2.5-7b, Search Engine=Wiki-182025.10 | 21.4 | 29.9 | |
| R1-instructModel Backbone=Qwen2.5-3b, Search Engine=Wiki-18, Model Variant=Instruct2025.10 | 20.4 | 29.7 | |
| CoTModel Backbone=Qwen2.5-3b, Search Engine=Wiki-182025.10 | 17.9 | 23.4 | |
| SFTModel Backbone=Qwen2.5-7b, Search Engine=Wiki-182025.10 | 12.4 | 17.8 | |
| Direct InferenceModel Backbone=Qwen2.5-7b, Search Engine=Wiki-182025.10 | 10.7 | 19.1 | |
| SFTModel Backbone=Qwen2.5-3b, Search Engine=Wiki-182025.10 | 10 | 18.1 | |
| Direct InferenceModel Backbone=Qwen2.5-3b, Search Engine=Wiki-182025.10 | 3.8 | 9.9 |