Single-Hop Question Answering on NQ (Natural Questions) in-domain (test)
31.45EMARPO
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ARPOModel=Qwen2.5-7B-Instruct, Retrieval Configuration=w/ Retrieval2026.02 | 31.45 | 1.93 | |
| SIGHTModel=Qwen2.5-7B-Instruct, Retrieval Configuration=w/ Retrieval2026.02 | 31.45 | 1.1 | |
| Tree-GRPOModel=Qwen2.5-7B-Instruct, Retrieval Configuration=w/ Retrieval2026.02 | 30.86 | 1.26 | |
| SIGHTModel=Qwen2.5-3B-Instruct, Retrieval Configuration=w/ Retrieval2026.02 | 29.3 | 1.19 | |
| ARPOModel=Qwen2.5-3B-Instruct, Retrieval Configuration=w/ Retrieval2026.02 | 28.32 | 2.93 | |
| Search-R1Model=Qwen2.5-7B-Instruct, Retrieval Configuration=w/ Retrieval2026.02 | 25.98 | 1.57 | |
| Search-R1Model=Qwen2.5-3B-Instruct, Retrieval Configuration=w/ Retrieval2026.02 | 25 | 1.69 | |
| Tree-GRPOModel=Qwen2.5-3B-Instruct, Retrieval Configuration=w/ Retrieval2026.02 | 23.24 | 1.47 | |
| Search-o1Model=Qwen2.5-7B-Instruct, Retrieval Configuration=w/ Retrieval2026.02 | 19.53 | 0.77 | |
| Search-o1Model=Qwen2.5-3B-Instruct, Retrieval Configuration=w/ Retrieval2026.02 | 16.99 | 0.9 | |
| GRPOModel=Qwen2.5-7B-Instruct, Retrieval Configuration=w/o Retrieval2026.02 | 16.21 | 0 | |
| GRPOModel=Qwen2.5-3B-Instruct, Retrieval Configuration=w/o Retrieval2026.02 | 14.65 | 0 | |
| Vanilla RAGModel=Qwen2.5-7B-Instruct, Retrieval Configuration=w/ Retrieval2026.02 | 9.96 | 1 | |
| ReActModel=Qwen2.5-7B-Instruct, Retrieval Configuration=w/ Retrieval2026.02 | 8.59 | 1.45 | |
| COTModel=Qwen2.5-3B-Instruct, Retrieval Configuration=w/o Retrieval2026.02 | 5.08 | 0 | |
| Direct ReasoningModel=Qwen2.5-3B-Instruct, Retrieval Configuration=w/o Retrieval2026.02 | 3.71 | 0 | |
| Vanilla RAGModel=Qwen2.5-3B-Instruct, Retrieval Configuration=w/ Retrieval2026.02 | 1.56 | 1 | |
| ReActModel=Qwen2.5-3B-Instruct, Retrieval Configuration=w/ Retrieval2026.02 | 0.2 | 1.89 | |
| Direct ReasoningModel=Qwen2.5-7B-Instruct, Retrieval Configuration=w/o Retrieval2026.02 | 0 | 0 | |
| COTModel=Qwen2.5-7B-Instruct, Retrieval Configuration=w/o Retrieval2026.02 | 0 | 0 |