Question Answering on NQ (test)
66.4EM AccuracyGolden
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| GoldenLLM=ChatGPT2024.06 | 66.4 | — | 76.86 | — | — | — | — | |
| FiE#Params=330M2023.05 | 58.4 | — | — | — | — | — | — | |
| COS + FiE#Params=330M2023.05 | 56.4 | — | — | — | — | — | — | |
| FiE (ours implementation)#Params=330M2023.05 | 56.3 | — | — | — | — | — | — | |
| R2-D2Number of Parameters=1.290B2022.12 | 55.9 | — | — | — | — | — | — | |
| R2-D2#Params=1.29B2023.05 | 55.9 | — | — | — | — | — | — | |
| Qwen3-Emb-4B + RerankerGenerator=T5Gemma22026.05 | 55.1 | — | 64.2 | — | — | — | — | |
| UnitedQANumber of Parameters=1.870B2022.12 | 54.7 | — | — | — | — | — | — | |
| ReAttNumber of Parameters=770M, Retriever=BM252022.12 | 54.7 | — | — | — | — | — | — | |
| UnitedQA#Params=1.87B2023.05 | 54.7 | — | — | — | — | — | — | |
| Qwen3-Emb-4BGenerator=T5Gemma22026.05 | 54.5 | — | 63.7 | — | — | — | — | |
| FiD-KDNumber of Parameters=990M2022.12 | 54.4 | — | — | — | — | — | — | |
| FiD-KD#Params=770M2023.05 | 54.4 | — | — | — | — | — | — | |
| ReAttNumber of Parameters=770M, Retriever=DPR2022.12 | 54 | — | — | — | — | — | — | |
| YONONumber of Parameters=440M, Retrieval Pre-training=true2022.12 | 53.2 | — | — | — | — | — | — | |
| YONO#Params=440M2023.05 | 53.2 | — | — | — | — | — | — | |
| EMDR2Number of Parameters=440M2022.12 | 52.5 | — | — | — | — | — | — | |
| EMDR2#Params=440M2023.05 | 52.5 | — | — | — | — | — | — | |
| BGEGenerator=T5Gemma22026.05 | 52.2 | — | 61.2 | — | — | — | — | |
| UnitedQA-E#Params=330M2023.05 | 51.8 | — | — | — | — | — | — | |
| FiDNumber of Parameters=990M2022.12 | 51.4 | — | — | — | — | — | — | |
| FiD#Params=770M2023.05 | 51.4 | — | — | — | — | — | — | |
| INTRAGenerator=T5Gemma22026.05 | 51.2 | — | 60.3 | — | — | — | — | |
| Hybrid RAGGenerator=T5Gemma22026.05 | 50.5 | — | 59.1 | — | — | — | — | |
| ITEM-AsLLM=ChatGPT, Iteration Rounds (m)=32024.06 | 49.69 | — | 60.18 | — | — | — | — | |
| ITEM-ARsLLM=ChatGPT, Iteration Rounds (m)=12024.06 | 49.52 | — | 59.64 | — | — | — | — | |
| ITEM-AsLLM=ChatGPT, Iteration Rounds (m)=12024.06 | 49.38 | — | 59.78 | — | — | — | — | |
| ITEM-ARsLLM=ChatGPT, Iteration Rounds (m)=32024.06 | 49.06 | — | 59.67 | — | — | — | — | |
| UJ-ImpALLM=ChatGPT2024.06 | 49.01 | — | 59.3 | — | — | — | — | |
| 5-samplingLLM=ChatGPT2024.06 | 48.9 | — | 58.97 | — | — | — | — | |
| VanillaLLM=ChatGPT2024.06 | 48.52 | — | 58.64 | — | — | — | — | |
| MaxSimGenerator=T5Gemma22026.05 | 48.4 | — | 57.8 | — | — | — | — | |
| UJ-ExpALLM=ChatGPT2024.06 | 47.72 | — | 58.01 | — | — | — | — | |
| MR-Search-BaseTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 47.7 | — | — | — | — | — | — | |
| INSPOBackbone=Qwen-2.5-7B, Category=Tool-integrated Methods2025.12 | 47.6 | — | — | — | — | — | — | |
| SD-Search-BaseTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 47 | — | — | — | — | — | — | |
| SD-Search-BaseBase Model=Qwen2.5-3B, Number of Seeds=52026.05 | 47 | — | — | — | — | — | — | |
| SD-Search-InstructTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 46.9 | — | — | — | — | — | — | |
| AutoRefine-BaseTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 46.7 | — | — | — | — | — | — | |
| DLLM=ChatGPT2024.06 | 46.54 | — | 57 | — | — | — | — | |
| AutoRefine-BaseBase Model=Qwen2.5-3B, Number of Seeds=52026.05 | 46.4 | — | — | — | — | — | — | |
| GoldenLLM=Mistral2024.06 | 46.09 | — | 62.59 | — | — | — | — | |
| RAGNumber of Parameters=220M2022.12 | 44.5 | — | — | — | — | — | — | |
| Thinker-InstructBase Model=Qwen2.5-3B, Number of Seeds=52026.05 | 44.1 | — | — | — | — | — | — | |
| Thinker-InstructTraining Pipeline=Synthesize → SFT, External Teacher=Qwen2.5-72B, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 43.9 | — | — | — | — | — | — | |
| Search-R1Backbone=Qwen-2.5-3B, Category=Tool-integrated Methods2025.12 | 43.8 | — | — | — | — | — | — | |
| INSPOBackbone=Qwen-2.5-3B, Category=Tool-integrated Methods2025.12 | 43.7 | — | — | — | — | — | — | |
| AutoRefine-InstructTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 43.6 | — | — | — | — | — | — | |
| Workflow-R1-SearchMulti-turn=true, Backbone=Qwen2.5-32B-Instruct2026.02 | 43.5 | — | — | — | — | — | — | |
| BM25Generator=T5Gemma22026.05 | 43.4 | — | 51.8 | — | — | — | — | |
| R2-Searcher2026.06 | 43.2 | — | 52.3 | — | — | — | — | |
| Search-R1 (GRPO)Multi-turn=true, Backbone=Qwen2.5-32B-Instruct2026.02 | 42.9 | — | — | — | — | — | — | |
| ReSearch-BaseTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 42.7 | — | — | — | — | — | — | |
| YONONumber of Parameters=440M, Retrieval Pre-training=false2022.12 | 42.4 | — | — | — | — | — | — | |
| Search-R1Backbone=Qwen-2.5-7B, Category=Tool-integrated Methods2025.12 | 42.3 | — | — | — | — | — | — | |
| Search-R1-BaseTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 42.1 | — | — | — | — | — | — | |
| AutoRefine2026.06 | 42.1 | — | 50.6 | — | — | — | — | |
| REALMNumber of Parameters=330M2022.12 | 40.4 | — | — | — | — | — | — | |
| ReSearch2026.06 | 40.2 | — | 48.7 | — | — | — | — | |
| Search-R1-InstructTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 39.7 | — | — | — | — | — | — | |
| Search-R1 (PPO)Multi-turn=true, Backbone=Qwen2.5-32B-Instruct2026.02 | 39.3 | — | — | — | — | — | — | |
| R3AGPipeline Type=Route2026.04 | 37.81 | — | 49.32 | — | — | — | — | |
| E5-largePipeline Type=Sequential2026.04 | 36.89 | — | 48 | — | — | — | — | |
| Oracle Single BestPipeline Type=Conditional2026.04 | 36.89 | — | 48 | — | — | — | — | |
| ReSearch-InstructTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 36.5 | — | — | — | — | — | — | |
| ECoRAG#tokens=6322025.06 | 36.48 | — | 49.81 | — | — | — | — | |
| Qwen3-Emb-0.6BGenerator=T5Gemma22026.05 | 36.4 | — | 44.6 | — | — | — | — | |
| E5-base-v2Pipeline Type=Sequential2026.04 | 36.12 | — | 47.35 | — | — | — | — | |
| standard RAG (100 documents)#tokens=139052025.06 | 36.09 | — | 50.18 | — | — | — | — | |
| SFTBackbone=Qwen-2.5-7B, Category=Tool-integrated Methods2025.12 | 36 | — | — | — | — | — | — | |
| IRCoTPipeline Type=Loop2026.04 | 35.81 | — | 47.95 | — | — | — | — | |
| LTRRPipeline Type=Route2026.04 | 35.76 | — | 46.9 | — | — | — | — | |
| CompAct#tokens=1062025.06 | 35.71 | — | 47.14 | — | — | — | — | |
| Adaptive-RAGPipeline Type=Conditional2026.04 | 35.12 | — | 45.55 | — | — | — | — | |
| RAGBackbone=Qwen-2.5-7B, Category=Tool-integrated Methods2025.12 | 34.9 | — | — | — | — | — | — | |
| TF-IDFGenerator=T5Gemma22026.05 | 34.9 | — | 42.9 | — | — | — | — | |
| RAGBackbone=Qwen-2.5-3B, Category=Tool-integrated Methods2025.12 | 34.8 | — | — | — | — | — | — | |
| Naive RAGTraining Pipeline=Zero-Shot, External Teacher=–, Retrieval Strategy=w/ Single-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 34.8 | — | — | — | — | — | — | |
| BGE-largePipeline Type=Sequential2026.04 | 34.51 | — | 45.27 | — | — | — | — | |
| ZeroSearch2026.06 | 34.5 | — | 43.5 | — | — | — | — | |
| Qwen3-embedding-4bPipeline Type=Sequential2026.04 | 34.49 | — | 45.8 | — | — | — | — | |
| ITEM-AsLLM=Mistral, Iteration Rounds (m)=32024.06 | 33.73 | — | 49.63 | — | — | — | — | |
| BGE-m3Pipeline Type=Sequential2026.04 | 33.52 | — | 43.49 | — | — | — | — | |
| ITEM-ARsLLM=Mistral, Iteration Rounds (m)=32024.06 | 33.4 | — | 49.27 | — | — | — | — | |
| ORQANumber of Parameters=330M2022.12 | 33.3 | — | — | — | — | — | — | |
| ITEM-ARsLLM=Mistral, Iteration Rounds (m)=12024.06 | 33.3 | — | 49.26 | — | — | — | — | |
| 5-samplingLLM=Mistral2024.06 | 33.24 | — | 48.84 | — | — | — | — | |
| ITEM-AsLLM=Mistral, Iteration Rounds (m)=12024.06 | 32.98 | — | 49 | — | — | — | — | |
| LongLLMLingua#tokens=13702025.06 | 32.96 | — | 45.32 | — | — | — | — | |
| Search-R12026.06 | 32.9 | — | 41.8 | — | — | — | — | |
| RECOMP (extractive)#tokens=6622025.06 | 32.85 | — | 44.54 | — | — | — | — | |
| UJ-ExpALLM=Mistral2024.06 | 32.76 | — | 48.46 | — | — | — | — | |
| GLaMModel Size=64B/64E, Evaluation Protocol=Few-shot, Shots=642021.12 | 32.5 | — | — | — | — | — | — | |
| RouterRetrieverPipeline Type=Route2026.04 | 32.44 | — | 43.31 | — | — | — | — | |
| Qwen3-embedding-0.6bPipeline Type=Sequential2026.04 | 32.35 | — | 42.71 | — | — | — | — | |
| closed-book#tokens=02025.06 | 31.88 | — | 44.1 | — | — | — | — | |
| SFTBackbone=Qwen-2.5-7B, Category=Model Inference2025.12 | 31.8 | — | — | — | — | — | — | |
| DLLM=Mistral2024.06 | 31.58 | — | 47.69 | — | — | — | — | |
| VanillaLLM=Mistral2024.06 | 31.16 | — | 47.43 | — | — | — | — | |
| UJ-ImpALLM=Mistral2024.06 | 30.67 | — | 46.83 | — | — | — | — |