Question Answering on 2Wiki (test)
61.8EM AccuracyICRL
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ICRLModel=Qwen2.5-14B2026.03 | 61.8 | — | — | |
| INTRAGenerator=T5Gemma22026.05 | 49.2 | — | 53.2 | |
| INTRARetrieval method=INTRA, Generator=T5Gemma2026.05 | 49.2 | — | — | |
| Thinker-InstructTraining Pipeline=Synthesize → SFT, External Teacher=Qwen2.5-72B, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 46.9 | — | — | |
| Qwen3-Emb-4B + RerankerGenerator=T5Gemma22026.05 | 46.8 | — | 50.8 | |
| Qwen3-Emb-4B + RerankerRetrieval method=Qwen3-Emb-4B + Reranker, Generator=T5Gemma2026.05 | 46.8 | — | — | |
| BGEGenerator=T5Gemma22026.05 | 46.1 | — | 49.6 | |
| BGERetrieval method=BGE, Generator=T5Gemma2026.05 | 46.1 | — | — | |
| Hybrid RAGGenerator=T5Gemma22026.05 | 46 | — | 49.9 | |
| Qwen3-Emb-4BGenerator=T5Gemma22026.05 | 46 | — | 50.1 | |
| Hybrid RAGRetrieval method=Hybrid RAG, Generator=T5Gemma2026.05 | 46 | — | — | |
| Qwen3-Emb-4BRetrieval method=Qwen3-Emb-4B, Generator=T5Gemma2026.05 | 46 | — | — | |
| Qwen3-Emb-0.6BGenerator=T5Gemma22026.05 | 45.7 | — | 49.8 | |
| Qwen3-Emb-0.6BRetrieval method=Qwen3-Emb-0.6B, Generator=T5Gemma2026.05 | 45.7 | — | — | |
| SD-Search-BaseTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 42 | — | — | |
| SD-Search-InstructTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 41.9 | — | — | |
| BM25Generator=T5Gemma22026.05 | 41.7 | — | 45.2 | |
| BM25Retrieval method=BM25, Generator=T5Gemma2026.05 | 41.7 | — | — | |
| MaxSimGenerator=T5Gemma22026.05 | 41.6 | — | 45.8 | |
| MaxSimRetrieval method=MaxSim, Generator=T5Gemma2026.05 | 41.6 | — | — | |
| MR-Search-BaseTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 40.1 | — | — | |
| AutoRefine-BaseTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 39.3 | — | — | |
| TF-IDFGenerator=T5Gemma22026.05 | 39 | — | 42.7 | |
| TF-IDFRetrieval method=TF-IDF, Generator=T5Gemma2026.05 | 39 | — | — | |
| AutoRefine-InstructTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 38 | — | — | |
| R2-Searcher2026.06 | 36.7 | — | 42.6 | |
| StepSearch-BaseTraining Pipeline=Pure RL, External Teacher=GPT-4o, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 33.9 | — | — | |
| AutoRefine2026.06 | 33.4 | — | 38.8 | |
| StepSearch-InstructTraining Pipeline=Pure RL, External Teacher=GPT-4o, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 32 | — | — | |
| Search-R1-InstructTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 31 | — | — | |
| Search-R12026.06 | 30.1 | — | 35.7 | |
| DirectModel=Qwen2.5-14B2026.03 | 28.2 | — | — | |
| R1-InstructTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/o Retrieval, Backbone=Qwen2.5-3B2026.05 | 27.5 | — | — | |
| Search-R1-BaseTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 27.4 | — | — | |
| ReSearch2026.06 | 27.4 | — | 33.6 | |
| ReSearch-BaseTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 27.2 | — | — | |
| ReSearch-InstructTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 27.2 | — | — | |
| R1-BaseTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/o Retrieval, Backbone=Qwen2.5-3B2026.05 | 26.8 | — | — | |
| CoTModel=Qwen2.5-14B2026.03 | 25.8 | — | — | |
| SFTTraining Pipeline=SFT, External Teacher=–, Retrieval Strategy=w/o Retrieval, Backbone=Qwen2.5-3B2026.05 | 24.8 | — | — | |
| Direct GenerationTraining Pipeline=Zero-Shot, External Teacher=–, Retrieval Strategy=w/o Retrieval, Backbone=Qwen2.5-3B2026.05 | 24.4 | — | — | |
| Naive RAGTraining Pipeline=Zero-Shot, External Teacher=–, Retrieval Strategy=w/ Single-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 22.6 | — | — | |
| Search-o1Training Pipeline=Prompting, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 21.8 | — | — | |
| ZeroSearch2026.06 | 21.3 | — | 25.9 | |
| Direct Inference2026.06 | 21.1 | — | 24.9 | |
| COT2026.06 | 19.4 | — | 22.9 | |
| IRCoTTraining Pipeline=Prompting, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 17.1 | — | — | |
| RAG2026.06 | 7.88 | — | 11 | |
| Search-o12026.06 | 7.74 | — | 12.8 | |
| IRCOT2026.06 | 7.15 | — | 13.6 | |
| AFHmode=Absolute2025.10 | 4.7 | 18.5 | — | |
| MASH w/ OTC-ST2025.10 | 4.6 | 32.5 | — | |
| MASH w/ OTC2025.10 | 4.1 | 31.3 | — | |
| MASH w/ EXP2025.10 | 3.8 | 32.2 | — | |
| Promptingshots=5-shot2025.10 | 3.6 | 10.9 | — | |
| DPO2025.10 | 3.3 | 31.6 | — | |
| AFHmode=Multisample2025.10 | 2.6 | 29.2 | — | |
| OTC2025.10 | 0.75 | 24.1 | — | |
| Ternary2025.10 | 0 | — | — |