Question Answering on Musique
26EMICRL
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ICRLModel=Qwen2.5-7B2026.03 | 26 | — | |
| SLATEBackbone=Qwen2.5-7B-Base2026.02 | 24.7 | — | |
| StepSearchBackbone=Qwen2.5-7B-Base, Training Data=MuSiQue (19k)2026.02 | 21.6 | — | |
| SLATEBackbone=Qwen2.5-3B-Base, Training Data=NQ+HotpotQA2026.02 | 21.6 | — | |
| ICRLModel=Qwen2.5-3B2026.03 | 20 | — | |
| ParallelSearchModel=Qwen2.5-7B2026.03 | 19.7 | — | |
| SEARCH-R1Backbone=Qwen2.5-7B-Base2026.02 | 19.6 | — | |
| LAnR-BaseRetrieval Strategy=w/ Multi-Hop Retrieval, Model Variant=Base2026.04 | 18.7 | — | |
| ZeroSearchModel=Qwen2.5-7B2026.03 | 18.4 | — | |
| StepSearchBackbone=Qwen2.5-3B-Base, Training Data=MuSiQue (19k), Knowledge Base=Wiki-182026.02 | 18.1 | — | |
| AutoRefine-InstructRetrieval Strategy=w/ Multi-Hop Retrieval, Model Variant=Instruct2026.04 | 16.9 | — | |
| AutoRefine-BaseRetrieval Strategy=w/ Multi-Hop Retrieval, Model Variant=Base2026.04 | 15.7 | — | |
| Search-R1Model=Qwen2.5-7B2026.03 | 14.6 | — | |
| ReSearchBackbone=Qwen2.5-7B-Base2026.02 | 14.3 | — | |
| Search-R1-InstructRetrieval Strategy=w/ Multi-Hop Retrieval, Model Variant=Instruct2026.04 | 12.4 | — | |
| Reject SamplingModel=Qwen2.5-7B2026.03 | 12.3 | — | |
| Search-R1Model=Qwen2.5-3B2026.03 | 10.3 | — | |
| ZeroSearchBackbone=Qwen2.5-7B-Base2026.02 | 10.2 | — | |
| ZeroSearchModel=Qwen2.5-3B2026.03 | 9.8 | — | |
| ReSearch-InstructRetrieval Strategy=w/ Multi-Hop Retrieval, Model Variant=Instruct2026.04 | 9.5 | — | |
| R1-baseModel=Qwen2.5-7B2026.03 | 8.3 | — | |
| AdaTIRBackbone=Qwen2.5-3b-Base2026.01 | 7.6 | 99 | |
| R1 (no search)Backbone=Qwen2.5-7B-Base2026.02 | 7.6 | — | |
| ReSearchBackbone=Qwen2.5-3B-Base2026.02 | 7.4 | — | |
| ReSearch-BaseRetrieval Strategy=w/ Multi-Hop Retrieval, Model Variant=Base2026.04 | 7.4 | — | |
| IRCoTModel=Qwen2.5-7B2026.03 | 7.2 | — | |
| R1-instructModel=Qwen2.5-7B2026.03 | 7.2 | — | |
| IRCoTBackbone=Qwen2.5-7B-Base2026.02 | 7.2 | — | |
| IRCoTModel=Qwen2.5-3B2026.03 | 6.7 | — | |
| IRCoTRetrieval Strategy=w/ Multi-Hop Retrieval2026.04 | 6.7 | — | |
| SFTModel=Qwen2.5-7B2026.03 | 6.6 | — | |
| Search-R1-BaseRetrieval Strategy=w/ Multi-Hop Retrieval, Model Variant=Base2026.04 | 6.6 | — | |
| RAGBackbone=Qwen2.5-7B-Base2026.02 | 6.1 | — | |
| R1-instructModel=Qwen2.5-3B2026.03 | 6 | — | |
| Reject SamplingModel=Qwen2.5-3B2026.03 | 5.9 | — | |
| IRCoTBackbone=Qwen2.5-3B-Base2026.02 | 5.9 | — | |
| Search-o1Model=Qwen2.5-7B2026.03 | 5.8 | — | |
| RAGModel=Qwen2.5-7B2026.03 | 5.8 | — | |
| ZeroSearchBackbone=Qwen2.5-3B-Base2026.02 | 5.6 | — | |
| Search-R1Backbone=Qwen2.5-3b-Base2026.01 | 5.4 | 100 | |
| Search-o1Model=Qwen2.5-3B2026.03 | 5.4 | — | |
| SFTBackbone=Qwen2.5-7B-Base2026.02 | 5.4 | — | |
| Search-o1Retrieval Strategy=w/ Multi-Hop Retrieval2026.04 | 5.4 | — | |
| Search-o1Backbone=Qwen2.5-7B-Base2026.02 | 5.3 | — | |
| RAGBackbone=Qwen2.5-3B-Base2026.02 | 5.1 | — | |
| R1 (no search)Backbone=Qwen2.5-3B-Base2026.02 | 5.1 | — | |
| SEARCH-R1Backbone=Qwen2.5-3B-Base, Training Data=NQ+HotpotQA2026.02 | 4.9 | — | |
| RAGModel=Qwen2.5-3B2026.03 | 4.7 | — | |
| Naive RAGRetrieval Strategy=w/ Single-Hop Retrieval2026.04 | 4.7 | — | |
| CoTBackbone=Qwen2.5-7B-Base2026.02 | 4.5 | — | |
| Search-o1Backbone=Qwen2.5-3B-Base2026.02 | 4.5 | — | |
| SFTModel=Qwen2.5-3B2026.03 | 4.4 | — | |
| SFTRetrieval Strategy=w/o Retrieval2026.04 | 4.4 | — | |
| SFTBackbone=Qwen2.5-3B-Base2026.02 | 3.9 | — | |
| CoTBackbone=Qwen2.5-3B-Base2026.02 | 3.2 | — | |
| DirectModel=Qwen2.5-7B2026.03 | 3.1 | — | |
| DirectBackbone=Qwen2.5-7B-Base2026.02 | 2.7 | — | |
| CoTModel=Qwen2.5-7B2026.03 | 2.2 | — | |
| DirectModel=Qwen2.5-3B2026.03 | 2 | — | |
| Direct GenerationRetrieval Strategy=w/o Retrieval2026.04 | 2 | — | |
| DirectBackbone=Qwen2.5-3B-Base2026.02 | 1.8 | — | |
| CoTModel=Qwen2.5-3B2026.03 | 0.2 | — |