General Question Answering on PopQA
45.2EMInForage-instruct
Evaluation results
| Method | Links | ||
|---|---|---|---|
| InForage-instructAlg.=PPO, Supervision=SFT + RL (Upper Bound)2026.04 | 45.2 | — | |
| Search-R1Base Model=Llama-3.2-3B-Instruct, Category=Outcome RL Baselines2025.12 | 44.8 | 1.5 | |
| ZeroSearch-instructAlg.=REINF., Supervision=RL (SFT-Free)2026.04 | 44.8 | — | |
| IKEABase Model=Qwen2.5-3B-Instruct, Category=Reward-Shaping Baselines2025.12 | 44.2 | 55.2 | |
| Agentic-RSearch Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 44.14 | — | |
| Search-R1Base Model=Qwen2.5-3B-Instruct, Category=Outcome RL Baselines2025.12 | 43.9 | 0 | |
| SubSearch-base (Ours)Alg.=GRPO, Supervision=RL (SFT-Free)2026.04 | 43.5 | — | |
| O2-Searcher-instructAlg.=GRPO, Supervision=SFT + RL (Upper Bound)2026.04 | 42.9 | — | |
| ADASEARCHBase Model=Qwen2.5-3B-Instruct, Category=Paper Method2025.12 | 42.8 | 58.8 | |
| LLM-EmbedderSearch Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 42.69 | — | |
| Agentic-RSearch Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 42.43 | — | |
| Naive ShapingBase Model=Qwen2.5-3B-Instruct, Category=Reward-Shaping Baselines2025.12 | 42.2 | 55.3 | |
| REPLUGSearch Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 42.11 | — | |
| SCARLetSearch Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 42.04 | — | |
| BGESearch Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 41.92 | — | |
| REPLUGSearch Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 41.85 | — | |
| E5Search Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 41.82 | — | |
| Awareness ShapingBase Model=Qwen2.5-3B-Instruct, Category=Reward-Shaping Baselines2025.12 | 41.8 | 61.3 | |
| Search-R1-baseAlg.=GRPO, Supervision=RL (SFT-Free)2026.04 | 41.3 | — | |
| Awareness ShapingBase Model=Llama-3.2-3B-Instruct, Category=Reward-Shaping Baselines2025.12 | 41.2 | 61.4 | |
| IKEABase Model=Llama-3.2-3B-Instruct, Category=Reward-Shaping Baselines2025.12 | 41.1 | 59.9 | |
| ADASEARCHBase Model=Llama-3.2-3B-Instruct, Category=Paper Method2025.12 | 40.4 | 62.3 | |
| Query Decomp-base (Ours)Alg.=GRPO, Supervision=RL (SFT-Free)2026.04 | 40.4 | — | |
| LLM-EmbedderSearch Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 40.05 | — | |
| Naive ShapingBase Model=Llama-3.2-3B-Instruct, Category=Reward-Shaping Baselines2025.12 | 39.3 | 56.3 | |
| Search-R1-instructAlg.=GRPO, Supervision=RL (SFT-Free)2026.04 | 39.1 | — | |
| Agentic-RSearch Agent=SimpleDeepSearcher, Search Agent Domain=out-of-domain2026.01 | 39.09 | — | |
| ZeroSearch-baseAlg.=GRPO, Supervision=RL (SFT-Free)2026.04 | 39 | — | |
| BGESearch Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 38.95 | — | |
| SCARLetSearch Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 38.73 | — | |
| RAGAlg.=–, Supervision=Baselines2026.04 | 38.7 | — | |
| E5Search Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 38.31 | — | |
| RAGBase Model=Qwen2.5-3B-Instruct, Category=Prompting Baselines2025.12 | 38.1 | 0 | |
| LLM-EmbedderSearch Agent=SimpleDeepSearcher, Search Agent Domain=out-of-domain2026.01 | 37.72 | — | |
| REPLUGSearch Agent=SimpleDeepSearcher, Search Agent Domain=out-of-domain2026.01 | 37.68 | — | |
| E5Search Agent=SimpleDeepSearcher, Search Agent Domain=out-of-domain2026.01 | 37.09 | — | |
| SCARLetSearch Agent=SimpleDeepSearcher, Search Agent Domain=out-of-domain2026.01 | 36.93 | — | |
| BGESearch Agent=SimpleDeepSearcher, Search Agent Domain=out-of-domain2026.01 | 36.83 | — | |
| R-Search-instruct (r.)Alg.=GRPO, Supervision=RL (SFT-Free)2026.04 | 36.5 | — | |
| RAGBase Model=Llama-3.2-3B-Instruct, Category=Prompting Baselines2025.12 | 35.7 | 0 | |
| Search-o1Alg.=–, Supervision=Baselines2026.04 | 26.2 | — | |
| RL w/o searchBase Model=Llama-3.2-3B-Instruct, Category=Outcome RL Baselines2025.12 | 23.1 | 23.1 | |
| RL w/o searchBase Model=Qwen2.5-3B-Instruct, Category=Outcome RL Baselines2025.12 | 17.3 | 17.3 | |
| R1-baseAlg.=PPO, Supervision=RL (SFT-Free)2026.04 | 17.3 | — | |
| Direct AnswerBase Model=Llama-3.2-3B-Instruct, Category=Prompting Baselines2025.12 | 16.4 | 16.4 | |
| CoTBase Model=Llama-3.2-3B-Instruct, Category=Prompting Baselines2025.12 | 15.8 | 15.8 | |
| CoTBase Model=Qwen2.5-3B-Instruct, Category=Prompting Baselines2025.12 | 13.5 | 13.5 | |
| Direct InferenceAlg.=–, Supervision=Baselines2026.04 | 10.8 | — | |
| SFTAlg.=–, Supervision=Baselines2026.04 | 10.4 | — | |
| Direct AnswerBase Model=Qwen2.5-3B-Instruct, Category=Prompting Baselines2025.12 | 7.8 | 7.8 | |
| CoTAlg.=–, Supervision=Baselines2026.04 | 0.5 | — |