General Question Answering on NQ (Natural Questions) (EM, LLM)
49.2Exact Match (EM)DAC
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DACBackbone=Qwen3-8B, FT=LoRA2026.06 | 49.2 | 62.8 | |
| Search-R1 (GRPO/O)Backbone=Qwen3-8B, FT=Full2026.06 | 48.8 | 62.2 | |
| Search-R1 (GRPO/OF)Backbone=Qwen3-8B, FT=Full2026.06 | 48.1 | 61.8 | |
| Search-R1 (PPO/OF)Backbone=Qwen3-8B, FT=Full2026.06 | 45.1 | 58.5 | |
| Search-R1 (GRPO/O)Backbone=Qwen2.5-7B-Instruct, FT=Full2026.06 | 43.9 | 57.2 | |
| Search-R1 (PPO/O)Backbone=Qwen3-8B, FT=Full2026.06 | 43.4 | 56.5 | |
| DACBackbone=Qwen2.5-7B-Instruct, FT=LoRA2026.06 | 41.9 | 59.4 | |
| Search-R1 (GRPO/OF)Backbone=Qwen2.5-7B-Instruct, FT=Full2026.06 | 39.6 | 57.9 | |
| Search-R1 (PPO/O)Backbone=Qwen2.5-7B-Instruct, FT=Full2026.06 | 39.2 | 55.9 | |
| Search-R1 (PPO/OF)Backbone=Qwen2.5-7B-Instruct, FT=Full2026.06 | 37.6 | 54.6 | |
| Naive RAGBackbone=Qwen2.5-7B-Instruct, FT=-2026.06 | 36.9 | 55.4 | |
| Naive RAGBackbone=Qwen3-8B, FT=-2026.06 | 35.9 | 54.9 | |
| IRCoTBackbone=Qwen3-8B, FT=-2026.06 | 35.6 | 57.9 | |
| Search-o1Backbone=Qwen3-8B, FT=-2026.06 | 32.8 | 56.3 | |
| Search-o1Backbone=Qwen2.5-7B-Instruct, FT=-2026.06 | 31.3 | 51.8 | |
| CoTBackbone=Qwen3-8B, FT=-2026.06 | 19.9 | 36 | |
| CoTBackbone=Qwen2.5-7B-Instruct, FT=-2026.06 | 19.3 | 33.7 | |
| DirectBackbone=Qwen2.5-7B-Instruct, FT=-2026.06 | 18.2 | 30.5 | |
| IRCoTBackbone=Qwen2.5-7B-Instruct, FT=-2026.06 | 16.6 | 46.8 | |
| DirectBackbone=Qwen3-8B, FT=-2026.06 | 16.3 | 32.5 |