Multi-Hop Question Answering on Bamboogle (EM, F1)
78.4EMSynPlanResearch-R1
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SynPlanResearch-R1Model Backbone=Qwen3-8B2026.03 | 78.4 | — | |
| SimpleDeepSearcherModel Backbone=Qwen3-8B2026.03 | 76.5 | — | |
| SynPlanResearch-R1Model Backbone=Qwen3-4B2026.03 | 75.3 | — | |
| Search-R1Model Backbone=Qwen3-8B2026.03 | 74.5 | — | |
| SimpleDeepSearcherModel Backbone=Qwen3-4B2026.03 | 73.8 | — | |
| Search-R1Model Backbone=Qwen3-4B2026.03 | 69.8 | — | |
| Rejection SamplingModel Backbone=Qwen3-8B2026.03 | 65.2 | — | |
| Rejection SamplingModel Backbone=Qwen3-4B2026.03 | 63.7 | — | |
| Search-o1Model Backbone=Qwen3-8B2026.03 | 54.6 | — | |
| Search-o1Model Backbone=Qwen3-4B2026.03 | 45.3 | — | |
| RAGModel Backbone=Qwen3-8B2026.03 | 32.3 | — | |
| GraphAnchorBackbone=Qwen3-32B2026.01 | 32.23 | 42.73 | |
| HierSearchBackbone=Qwen2.5-7B-Instruct2026.01 | 32 | — | |
| O2-SearcherBackbone=Qwen2.5-7B-Instruct2026.01 | 30.89 | — | |
| Dep-SearchBackbone=Qwen2.5-7B-Instruct2026.01 | 30.56 | — | |
| Search-O1Backbone=Qwen2.5-7B-Instruct2026.01 | 30.4 | — | |
| RA-ISFBackbone=Qwen2.5-7B-Instruct2026.01 | 28.9 | — | |
| R1-SearcherBackbone=Qwen2.5-7B-Instruct2026.01 | 28.89 | — | |
| GraphAnchorBackbone=Llama3.1-8B-Instruct2026.01 | 28.69 | 36.55 | |
| IRCoTBackbone=Qwen2.5-7B-Instruct2026.01 | 27.78 | — | |
| DeepNoteBackbone=Llama3.1-8B-Instruct2026.01 | 27.2 | 37.46 | |
| GraphAnchorBackbone=Qwen2.5-14B-Instruct2026.01 | 27.2 | 35.22 | |
| RAGModel Backbone=Qwen3-4B2026.03 | 27.2 | — | |
| Search-R1Backbone=Qwen2.5-7B-Instruct2026.01 | 26.39 | — | |
| ZeroSearchBackbone=Qwen2.5-7B-Instruct2026.01 | 25.56 | — | |
| GraphAnchorBackbone=Qwen2.5-7B-Instruct2026.01 | 24.8 | 34.25 | |
| Dep-SearchBackbone=Qwen2.5-3B-Instruct2026.01 | 24 | — | |
| DeepNoteBackbone=Qwen2.5-7B-Instruct2026.01 | 24 | 32.66 | |
| IRCoTBackbone=Llama3.1-8B-Instruct2026.01 | 23.2 | 32.46 | |
| HierSearchBackbone=Qwen2.5-3B-Instruct2026.01 | 22.56 | — | |
| Search-O1Backbone=Qwen2.5-3B-Instruct2026.01 | 22.4 | — | |
| O2-SearcherBackbone=Qwen2.5-3B-Instruct2026.01 | 21.44 | — | |
| Search-R1Backbone=Qwen2.5-7B-Instruct2026.01 | 20 | 25.13 | |
| Iter-RetGenBackbone=Llama3.1-8B-Instruct2026.01 | 20 | 26.09 | |
| R1-SearcherBackbone=Qwen2.5-3B-Instruct2026.01 | 18.5 | — | |
| Direct InferenceModel Backbone=Qwen3-8B2026.03 | 17.5 | — | |
| Directly InferenceBackbone=Qwen2.5-3B-Instruct2026.01 | 16.8 | — | |
| Iter-RetGenBackbone=Qwen2.5-7B-Instruct2026.01 | 16.8 | 24.82 | |
| Vanilla RAGBackbone=Qwen3-32B2026.01 | 16.8 | 25.91 | |
| ZeroSearchBackbone=Qwen2.5-3B-Instruct2026.01 | 16.67 | — | |
| Vanilla RAGBackbone=Llama3.1-8B-Instruct2026.01 | 16 | 24.4 | |
| Vanilla RAGBackbone=Qwen2.5-7B-Instruct2026.01 | 15.2 | — | |
| Vanilla RAGBackbone=Qwen2.5-14B-Instruct2026.01 | 15.2 | 26.74 | |
| Vanilla RAGBackbone=Qwen2.5-3B-Instruct2026.01 | 14.4 | — | |
| Directly InferenceBackbone=Qwen2.5-7B-Instruct2026.01 | 14.4 | — | |
| Vanilla RAGBackbone=Qwen2.5-7B-Instruct2026.01 | 13.6 | 21.62 | |
| IRCoTBackbone=Qwen2.5-7B-Instruct2026.01 | 12.8 | 21.15 | |
| Search-R1Backbone=Qwen2.5-3B-Instruct2026.01 | 12.5 | — | |
| Direct InferenceModel Backbone=Qwen3-4B2026.03 | 11.1 | — | |
| RA-ISFBackbone=Qwen2.5-3B-Instruct2026.01 | 6.2 | — | |
| IRCoTBackbone=Qwen2.5-3B-Instruct2026.01 | 5.56 | — |