Multi-Hop Question Answering on Bamboogle (EM, F1aware)
42.4EMSearch-R1
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Search-R1Base Model=Llama-3.2-3B-Instruct, Category=Outcome RL Baselines2025.12 | 42.4 | 0 | |
| ADASEARCHBase Model=Llama-3.2-3B-Instruct, Category=Paper Method2025.12 | 42.4 | 64.2 | |
| IKEABase Model=Llama-3.2-3B-Instruct, Category=Reward-Shaping Baselines2025.12 | 40.8 | 63.1 | |
| Naive ShapingBase Model=Llama-3.2-3B-Instruct, Category=Reward-Shaping Baselines2025.12 | 36.8 | 55.9 | |
| CoTBase Model=Llama-3.2-3B-Instruct, Category=Prompting Baselines2025.12 | 35.2 | 35.2 | |
| RL w/o searchBase Model=Llama-3.2-3B-Instruct, Category=Outcome RL Baselines2025.12 | 34.4 | 34.4 | |
| Awareness ShapingBase Model=Llama-3.2-3B-Instruct, Category=Reward-Shaping Baselines2025.12 | 32 | 58.1 | |
| Search-R1Base Model=Qwen2.5-3B-Instruct, Category=Outcome RL Baselines2025.12 | 29.6 | 0 | |
| ADASEARCHBase Model=Qwen2.5-3B-Instruct, Category=Paper Method2025.12 | 28 | 50 | |
| Naive ShapingBase Model=Qwen2.5-3B-Instruct, Category=Reward-Shaping Baselines2025.12 | 24.8 | 43.8 | |
| CoTBase Model=Qwen2.5-3B-Instruct, Category=Prompting Baselines2025.12 | 24 | 24 | |
| RL w/o searchBase Model=Qwen2.5-3B-Instruct, Category=Outcome RL Baselines2025.12 | 23.2 | 23.2 | |
| IKEABase Model=Qwen2.5-3B-Instruct, Category=Reward-Shaping Baselines2025.12 | 23.2 | 46 | |
| Awareness ShapingBase Model=Qwen2.5-3B-Instruct, Category=Reward-Shaping Baselines2025.12 | 22.4 | 41.1 | |
| RAGBase Model=Llama-3.2-3B-Instruct, Category=Prompting Baselines2025.12 | 12.8 | 0 | |
| Direct AnswerBase Model=Llama-3.2-3B-Instruct, Category=Prompting Baselines2025.12 | 8.8 | 8.8 | |
| RAGBase Model=Qwen2.5-3B-Instruct, Category=Prompting Baselines2025.12 | 8 | 0 | |
| Direct AnswerBase Model=Qwen2.5-3B-Instruct, Category=Prompting Baselines2025.12 | 5.6 | 5.6 |