General Question Answering on NQ (Natural Questions) (EM, F1aware)
46.1EMSearch-R1
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Search-R1Base Model=Llama-3.2-3B-Instruct, Category=Outcome RL Baselines2025.12 | 46.1 | 1.7 | |
| ADASEARCHBase Model=Llama-3.2-3B-Instruct, Category=Paper Method2025.12 | 43.5 | 62.7 | |
| Search-R1Base Model=Qwen2.5-3B-Instruct, Category=Outcome RL Baselines2025.12 | 42.7 | 0 | |
| GraphRAG-RouterSystem=Our Method2026.03 | 42.6 | — | |
| IKEABase Model=Llama-3.2-3B-Instruct, Category=Reward-Shaping Baselines2025.12 | 42.2 | 58.6 | |
| IKEABase Model=Qwen2.5-3B-Instruct, Category=Reward-Shaping Baselines2025.12 | 41.4 | 37.7 | |
| Awareness ShapingBase Model=Llama-3.2-3B-Instruct, Category=Reward-Shaping Baselines2025.12 | 41.4 | 58.8 | |
| Naive ShapingBase Model=Llama-3.2-3B-Instruct, Category=Reward-Shaping Baselines2025.12 | 41.2 | 57.6 | |
| Search-R1System=RL-based Agent2026.03 | 40.3 | — | |
| Router-R1System=RL-based Agent2026.03 | 38.6 | — | |
| RL w/o searchBase Model=Llama-3.2-3B-Instruct, Category=Outcome RL Baselines2025.12 | 38.2 | 38.2 | |
| ADASEARCHBase Model=Qwen2.5-3B-Instruct, Category=Paper Method2025.12 | 37.9 | 49 | |
| Naive ShapingBase Model=Qwen2.5-3B-Instruct, Category=Reward-Shaping Baselines2025.12 | 37.8 | 41.6 | |
| Awareness ShapingBase Model=Qwen2.5-3B-Instruct, Category=Reward-Shaping Baselines2025.12 | 37.7 | 45.9 | |
| GraphSearchSystem=Training-free Agent2026.03 | 36.8 | — | |
| HippoRAG2System=RAG-Based2026.03 | 35.3 | — | |
| Search-o1System=Training-free Agent2026.03 | 34.8 | — | |
| RAGBase Model=Qwen2.5-3B-Instruct, Category=Prompting Baselines2025.12 | 31.7 | 0 | |
| LinearRAGSystem=RAG-Based2026.03 | 30.2 | — | |
| GraphRAGSystem=RAG-Based2026.03 | 30.1 | — | |
| RAGBase Model=Llama-3.2-3B-Instruct, Category=Prompting Baselines2025.12 | 29.4 | 0 | |
| Vanilla RAGSystem=RAG-Based2026.03 | 29.4 | — | |
| CoTSystem=Basic LLM2026.03 | 28.6 | — | |
| RAPTORSystem=RAG-Based2026.03 | 27.1 | — | |
| HyperGraphRAGSystem=RAG-Based2026.03 | 26.8 | — | |
| Graph-R1System=RL-based Agent2026.03 | 26.7 | — | |
| CoTBase Model=Llama-3.2-3B-Instruct, Category=Prompting Baselines2025.12 | 25 | 25 | |
| RL w/o searchBase Model=Qwen2.5-3B-Instruct, Category=Outcome RL Baselines2025.12 | 21.7 | 21.7 | |
| Direct AnswerBase Model=Llama-3.2-3B-Instruct, Category=Prompting Baselines2025.12 | 18.8 | 18.8 | |
| CoTBase Model=Qwen2.5-3B-Instruct, Category=Prompting Baselines2025.12 | 14.1 | 14.1 | |
| Direct InferSystem=Basic LLM2026.03 | 10.7 | — | |
| Direct AnswerBase Model=Qwen2.5-3B-Instruct, Category=Prompting Baselines2025.12 | 8.1 | 8.1 |