Multi-Hop Question Answering on Bamboogle (EM, LLM)
49.6EMSearch-R1 (PPO/OF)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Search-R1 (PPO/OF)Backbone=Qwen3-8B, FT=Full2026.06 | 49.6 | 59.2 | |
| DACBackbone=Qwen3-8B, FT=LoRA2026.06 | 49.6 | 61.6 | |
| Search-R1 (GRPO/O)Backbone=Qwen3-8B, FT=Full2026.06 | 48.8 | 56.8 | |
| Search-R1 (GRPO/OF)Backbone=Qwen3-8B, FT=Full2026.06 | 48.8 | 59.2 | |
| Search-R1 (PPO/O)Backbone=Qwen3-8B, FT=Full2026.06 | 48 | 58.4 | |
| DACBackbone=Qwen2.5-7B-Instruct, FT=LoRA2026.06 | 41.6 | 52 | |
| Search-R1 (PPO/O)Backbone=Qwen2.5-7B-Instruct, FT=Full2026.06 | 40.8 | 53.6 | |
| Search-R1 (GRPO/O)Backbone=Qwen2.5-7B-Instruct, FT=Full2026.06 | 40 | 50.4 | |
| Search-R1 (PPO/OF)Backbone=Qwen2.5-7B-Instruct, FT=Full2026.06 | 37.6 | 48 | |
| IRCoTBackbone=Qwen3-8B, FT=-2026.06 | 36.8 | 47.2 | |
| Search-R1 (GRPO/OF)Backbone=Qwen2.5-7B-Instruct, FT=Full2026.06 | 35.2 | 45.6 | |
| Search-o1Backbone=Qwen2.5-7B-Instruct, FT=-2026.06 | 33.6 | 41.6 | |
| CoTBackbone=Qwen3-8B, FT=-2026.06 | 32.8 | 40.8 | |
| Search-o1Backbone=Qwen3-8B, FT=-2026.06 | 32 | 48 | |
| Naive RAGBackbone=Qwen3-8B, FT=-2026.06 | 26.4 | 34.4 | |
| CoTBackbone=Qwen2.5-7B-Instruct, FT=-2026.06 | 25.6 | 30.4 | |
| Naive RAGBackbone=Qwen2.5-7B-Instruct, FT=-2026.06 | 24 | 28 | |
| IRCoTBackbone=Qwen2.5-7B-Instruct, FT=-2026.06 | 15.2 | 26.4 | |
| DirectBackbone=Qwen3-8B, FT=-2026.06 | 10.4 | 12.8 | |
| DirectBackbone=Qwen2.5-7B-Instruct, FT=-2026.06 | 9.6 | 11.2 |