Question Answering on TriviaQA (test) (EM)
92.1EMSearch-R1 (EM)
Evaluation Results
| Method | Links | |
|---|---|---|
| Search-R1 (EM)Backbone=Qwen3-4B-Instruct-2507, Supervision Strategy=W. Gold Supervision2026.04 | 92.1 | |
| TTRLBackbone=Qwen3-4B-Instruct-2507, Supervision Strategy=W.O. Gold Supervision2026.04 | 90.2 | |
| CCSBackbone=Qwen3-4B-Instruct-2507, Supervision Strategy=W.O. Gold Supervision2026.04 | 90.2 | |
| CCSBackbone=Qwen3-32B, Supervision Strategy=W.O. Gold Supervision2026.04 | 89.6 | |
| CJBackbone=Qwen3-4B-Instruct-2507, Supervision Strategy=W.O. Gold Supervision2026.04 | 89.3 | |
| RLIFBackbone=Qwen3-32B, Supervision Strategy=W.O. Gold Supervision2026.04 | 88.7 | |
| Search-o1Backbone=Qwen3-32B, Supervision Strategy=Model Inference2026.04 | 88.1 | |
| CCSBackbone=Qwen2.5-7B-Instruct, Supervision Strategy=W.O. Gold Supervision2026.04 | 88 | |
| Search-R1 (EM)Backbone=Qwen3-32B, Supervision Strategy=W. Gold Supervision2026.04 | 88 | |
| RLIFBackbone=Qwen3-4B-Instruct-2507, Supervision Strategy=W.O. Gold Supervision2026.04 | 87.9 | |
| Search-R1 (EM)Backbone=Qwen2.5-7B-Instruct, Supervision Strategy=W. Gold Supervision2026.04 | 87.2 | |
| TTRLBackbone=Qwen3-32B, Supervision Strategy=W.O. Gold Supervision2026.04 | 87 | |
| CJBackbone=Qwen2.5-7B-Instruct, Supervision Strategy=W.O. Gold Supervision2026.04 | 86.5 | |
| RLIFBackbone=Qwen2.5-7B-Instruct, Supervision Strategy=W.O. Gold Supervision2026.04 | 84.6 | |
| RAGBackbone=Qwen3-32B, Supervision Strategy=Model Inference2026.04 | 83.8 | |
| RAGBackbone=Qwen2.5-7B-Instruct, Supervision Strategy=Model Inference2026.04 | 80.5 | |
| RAGBackbone=Qwen3-4B-Instruct-2507, Supervision Strategy=Model Inference2026.04 | 80.3 | |
| IRCoTBackbone=Qwen3-4B-Instruct-2507, Supervision Strategy=Model Inference2026.04 | 77 | |
| IRCoTBackbone=Qwen2.5-7B-Instruct, Supervision Strategy=Model Inference2026.04 | 76.1 | |
| ICRLModel=Qwen2.5-14B2026.03 | 75 | |
| IRCoTBackbone=Qwen3-32B, Supervision Strategy=Model Inference2026.04 | 69.7 | |
| SFTBackbone=Qwen3-32B, Supervision Strategy=W. Gold Supervision2026.04 | 67 | |
| INSPOBackbone=Qwen-2.5-7B, Category=Tool-integrated Methods2025.12 | 64.6 | |
| MR-Search-BaseTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 63.5 | |
| SD-Search-BaseTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 62.4 | |
| SD-Search-InstructTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 62.4 | |
| SD-Search-BaseBase Model=Qwen2.5-3B, Number of Seeds=52026.05 | 62.4 | |
| AutoRefine-BaseBase Model=Qwen2.5-3B, Number of Seeds=52026.05 | 62.1 | |
| PersonalAILLM=DeepSeek V3, Retrieval algorithm=hybrid of BeamSearch and WaterCircles, Graph restriction=no restrictions applied2025.06 | 62 | |
| AutoRefine-BaseTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 62 | |
| Search-R1Backbone=Qwen-2.5-3B, Category=Tool-integrated Methods2025.12 | 59.9 | |
| Thinker-InstructTraining Pipeline=Synthesize → SFT, External Teacher=Qwen2.5-72B, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 59.8 | |
| ReSearch-BaseTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 59.7 | |
| AutoRefine-InstructTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 59.7 | |
| Thinker-InstructBase Model=Qwen2.5-3B, Number of Seeds=52026.05 | 59.4 | |
| SFTBackbone=Qwen-2.5-7B, Category=Tool-integrated Methods2025.12 | 59.2 | |
| Search-o1Backbone=Qwen2.5-7B-Instruct, Supervision Strategy=Model Inference2026.04 | 58.8 | |
| INSPOBackbone=Qwen-2.5-3B, Category=Tool-integrated Methods2025.12 | 58.7 | |
| SFTBackbone=Qwen2.5-7B-Instruct, Supervision Strategy=W. Gold Supervision2026.04 | 58.7 | |
| RAGBackbone=Qwen-2.5-7B, Category=Tool-integrated Methods2025.12 | 58.5 | |
| Search-R1-BaseTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 58.3 | |
| Search-R1Backbone=Qwen-2.5-7B, Category=Tool-integrated Methods2025.12 | 57.5 | |
| ReSearch-InstructTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 57.1 | |
| Search-R1-InstructTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 56.5 | |
| CoTModel=Qwen2.5-14B2026.03 | 56.4 | |
| CJBackbone=Qwen3-32B, Supervision Strategy=W.O. Gold Supervision2026.04 | 55.6 | |
| RAGBackbone=Qwen-2.5-3B, Category=Tool-integrated Methods2025.12 | 54.4 | |
| Naive RAGTraining Pipeline=Zero-Shot, External Teacher=–, Retrieval Strategy=w/ Single-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 54.4 | |
| GRPOBackbone=Qwen-2.5-7B, Category=Model Inference2025.12 | 53.9 | |
| DirectModel=Qwen2.5-14B2026.03 | 52 | |
| TTRLBackbone=Qwen2.5-7B-Instruct, Supervision Strategy=W.O. Gold Supervision2026.04 | 50 | |
| Search-o1Backbone=Qwen3-4B-Instruct-2507, Supervision Strategy=Model Inference2026.04 | 48.9 | |
| SFTBackbone=Qwen-2.5-3B, Category=Tool-integrated Methods2025.12 | 48.8 | |
| IRCoTBackbone=Qwen-2.5-7B, Category=Tool-integrated Methods2025.12 | 47.8 | |
| SFTBackbone=Qwen3-4B-Instruct-2507, Supervision Strategy=W. Gold Supervision2026.04 | 47.4 | |
| Search-o1Backbone=Qwen-2.5-3B, Category=Tool-integrated Methods2025.12 | 47.2 | |
| Search-o1Training Pipeline=Prompting, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 47.2 | |
| PersonalAILLM=Llama3.1 8B, Retrieval algorithm=BeamSearch, Graph restriction=episodic vertices excluded2025.06 | 47 | |
| GRPOBackbone=Qwen-2.5-3B, Category=Model Inference2025.12 | 45.5 | |
| R1-BaseTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/o Retrieval, Backbone=Qwen2.5-3B2026.05 | 45.5 | |
| T2MBackbone=LLaDA2.1-mini (16B MoE), Decoding Strategy=greedy decoding, Temperature=0, block length=32, LOWPROB parameters=τ=0.3, Cmax=1, ρmax=0.252026.04 | 44.98 | |
| R1-InstructTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/o Retrieval, Backbone=Qwen2.5-3B2026.05 | 44.9 | |
| Search-o1Backbone=Qwen-2.5-7B, Category=Tool-integrated Methods2025.12 | 44.3 | |
| T2TBackbone=LLaDA2.1-mini (16B MoE), Decoding Strategy=greedy decoding, Temperature=0, block length=322026.04 | 43.71 | |
| DirectBackbone=Qwen-2.5-7B, Category=Model Inference2025.12 | 40.8 | |
| SFTBackbone=Qwen-2.5-7B, Category=Model Inference2025.12 | 35.4 | |
| CoTBackbone=Qwen3-32B, Supervision Strategy=Model Inference2026.04 | 32.2 | |
| IRCoTBackbone=Qwen-2.5-3B, Category=Tool-integrated Methods2025.12 | 31.2 | |
| IRCoTTraining Pipeline=Prompting, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 31.2 | |
| Direct InferenceBackbone=Qwen3-32B, Supervision Strategy=Model Inference2026.04 | 30.8 | |
| SFTBackbone=Qwen-2.5-3B, Category=Model Inference2025.12 | 29.2 | |
| SFTTraining Pipeline=SFT, External Teacher=–, Retrieval Strategy=w/o Retrieval, Backbone=Qwen2.5-3B2026.05 | 29.2 | |
| DirectBackbone=Qwen-2.5-3B, Category=Model Inference2025.12 | 28.8 | |
| Direct GenerationTraining Pipeline=Zero-Shot, External Teacher=–, Retrieval Strategy=w/o Retrieval, Backbone=Qwen2.5-3B2026.05 | 28.8 | |
| PersonalAILLM=Qwen2.5 7B, Retrieval algorithm=BeamSearch, Graph restriction=episodic vertices excluded2025.06 | 20 | |
| PersonalAILLM=DeepSeek R1 7B, Retrieval algorithm=A*, Graph restriction=episodic vertices excluded2025.06 | 18 | |
| CoTBackbone=Qwen3-4B-Instruct-2507, Supervision Strategy=Model Inference2026.04 | 14.2 | |
| CoTBackbone=Qwen2.5-7B-Instruct, Supervision Strategy=Model Inference2026.04 | 13.9 | |
| Direct InferenceBackbone=Qwen2.5-7B-Instruct, Supervision Strategy=Model Inference2026.04 | 11.1 | |
| Direct InferenceBackbone=Qwen3-4B-Instruct-2507, Supervision Strategy=Model Inference2026.04 | 9.1 |