Multi-Hop Question Answering on Bamboogle (test) (Exact Match (EM))
74.2Exact Match (EM)Search-R1 (EM)
Evaluation results
| Method | Links | |
|---|---|---|
| Search-R1 (EM)Backbone=Qwen3-32B, Supervision Strategy=W. Gold Supervision2026.04 | 74.2 | |
| CJBackbone=Qwen3-32B, Supervision Strategy=W.O. Gold Supervision2026.04 | 73.1 | |
| CCSBackbone=Qwen3-32B, Supervision Strategy=W.O. Gold Supervision2026.04 | 71.2 | |
| CCSBackbone=Qwen3-4B-Instruct-2507, Supervision Strategy=W.O. Gold Supervision2026.04 | 63.2 | |
| CCSBackbone=Qwen2.5-7B-Instruct, Supervision Strategy=W.O. Gold Supervision2026.04 | 60.8 | |
| Search-R1 (EM)Backbone=Qwen3-4B-Instruct-2507, Supervision Strategy=W. Gold Supervision2026.04 | 60.8 | |
| Search-o1Backbone=Qwen3-32B, Supervision Strategy=Model Inference2026.04 | 58.4 | |
| ReCal+Routing Strategy=RL-based Routing, Task Decomposition=false2026.06 | 58.4 | |
| Search-R1 (EM)Backbone=Qwen2.5-7B-Instruct, Supervision Strategy=W. Gold Supervision2026.04 | 56.8 | |
| ReCalRouting Strategy=RL-based Routing, Task Decomposition=false2026.06 | 56.8 | |
| CJBackbone=Qwen3-4B-Instruct-2507, Supervision Strategy=W.O. Gold Supervision2026.04 | 55.2 | |
| CJBackbone=Qwen2.5-7B-Instruct, Supervision Strategy=W.O. Gold Supervision2026.04 | 54.4 | |
| TTRLBackbone=Qwen2.5-7B-Instruct, Supervision Strategy=W.O. Gold Supervision2026.04 | 54 | |
| TTRLBackbone=Qwen3-32B, Supervision Strategy=W.O. Gold Supervision2026.04 | 54 | |
| Search-o1Backbone=Qwen2.5-7B-Instruct, Supervision Strategy=Model Inference2026.04 | 53.6 | |
| RLIFBackbone=Qwen3-32B, Supervision Strategy=W.O. Gold Supervision2026.04 | 52.8 | |
| RLIFBackbone=Qwen3-4B-Instruct-2507, Supervision Strategy=W.O. Gold Supervision2026.04 | 52 | |
| RouterDCRouting Strategy=Heuristic & Discriminative Routing, Task Decomposition=false2026.06 | 50.4 | |
| TTRLBackbone=Qwen3-4B-Instruct-2507, Supervision Strategy=W.O. Gold Supervision2026.04 | 48.2 | |
| Largest LLMRouting Strategy=Heuristic & Discriminative Routing, Task Decomposition=false2026.06 | 48 | |
| Prompt LLMRouting Strategy=Heuristic & Discriminative Routing, Task Decomposition=true2026.06 | 47.2 | |
| Prompt LLMRouting Strategy=Heuristic & Discriminative Routing, Task Decomposition=false2026.06 | 44.8 | |
| GraphRouterRouting Strategy=Heuristic & Discriminative Routing, Task Decomposition=false2026.06 | 44.8 | |
| Router-R1Routing Strategy=RL-based Routing, Task Decomposition=false2026.06 | 42.4 | |
| RLIFBackbone=Qwen2.5-7B-Instruct, Supervision Strategy=W.O. Gold Supervision2026.04 | 40.8 | |
| CoTBackbone=Qwen3-4B-Instruct-2507, Supervision Strategy=Model Inference2026.04 | 40.8 | |
| RAGBackbone=Qwen3-32B, Supervision Strategy=Model Inference2026.04 | 40 | |
| CoTBackbone=Qwen2.5-7B-Instruct, Supervision Strategy=Model Inference2026.04 | 38.4 | |
| Search-o1Backbone=Qwen3-4B-Instruct-2507, Supervision Strategy=Model Inference2026.04 | 38.4 | |
| KNN RouterRouting Strategy=Heuristic & Discriminative Routing, Task Decomposition=true2026.06 | 38.4 | |
| KNN RouterRouting Strategy=Heuristic & Discriminative Routing, Task Decomposition=false2026.06 | 36 | |
| MLP RouterRouting Strategy=Heuristic & Discriminative Routing, Task Decomposition=false2026.06 | 36 | |
| CoTBackbone=Qwen3-32B, Supervision Strategy=Model Inference2026.04 | 33.6 | |
| RAGBackbone=Qwen3-4B-Instruct-2507, Supervision Strategy=Model Inference2026.04 | 31.2 | |
| IRCoTBackbone=Qwen3-32B, Supervision Strategy=Model Inference2026.04 | 31.2 | |
| RouteLLMRouting Strategy=Heuristic & Discriminative Routing, Task Decomposition=false2026.06 | 31.2 | |
| IRCoTBackbone=Qwen2.5-7B-Instruct, Supervision Strategy=Model Inference2026.04 | 30.4 | |
| RAGBackbone=Qwen2.5-7B-Instruct, Supervision Strategy=Model Inference2026.04 | 30.4 | |
| SFTBackbone=Qwen3-32B, Supervision Strategy=W. Gold Supervision2026.04 | 28.8 | |
| Search-R1Routing Strategy=No Routing, Task Decomposition=false2026.06 | 27.2 | |
| IRCoTBackbone=Qwen3-4B-Instruct-2507, Supervision Strategy=Model Inference2026.04 | 26.4 | |
| Direct InferenceBackbone=Qwen3-4B-Instruct-2507, Supervision Strategy=Model Inference2026.04 | 25.6 | |
| SFTBackbone=Qwen2.5-7B-Instruct, Supervision Strategy=W. Gold Supervision2026.04 | 22.4 | |
| Direct InferenceBackbone=Qwen3-32B, Supervision Strategy=Model Inference2026.04 | 22.4 | |
| CoTRouting Strategy=No Routing, Task Decomposition=false2026.06 | 22.4 | |
| RAGRouting Strategy=No Routing, Task Decomposition=false2026.06 | 22.4 | |
| Direct InferenceBackbone=Qwen2.5-7B-Instruct, Supervision Strategy=Model Inference2026.04 | 12 | |
| SFTBackbone=Qwen3-4B-Instruct-2507, Supervision Strategy=W. Gold Supervision2026.04 | 12 | |
| SFTRouting Strategy=No Routing, Task Decomposition=false2026.06 | 11.2 | |
| VanillaRouting Strategy=No Routing, Task Decomposition=false2026.06 | 4 |