Multi-hop Question Answering on SimpleQA
18.8Pass@1R1-70B
Evaluation results
| Method | Links | |
|---|---|---|
| R1-70B2025.05 | 18.8 | |
| o3-mini2025.05 | 13.8 | |
| Qwen2.5-72B2025.05 | 11.7 | |
| Qwen2.5-32B + cotProtocol=CoT2025.05 | 9.9 | |
| QwQ-32B2025.05 | 9.7 | |
| Qwen2.5-32B + rtProtocol=rt2025.05 | 9.4 | |
| Qwen2.5-32B2025.05 | 8.7 | |
| Qwen2.5-14B + rtProtocol=rt2025.05 | 8.2 | |
| Qwen2.5-14B + cotProtocol=CoT2025.05 | 8 | |
| Qwen2.5-14B + fs1Protocol=fs12025.05 | 7.9 | |
| Qwen2.5-32B + fs1Protocol=fs12025.05 | 7.8 | |
| Qwen2.5-7B2025.05 | 7 | |
| Qwen2.5-14B2025.05 | 6.8 | |
| Qwen2.5-7B + rtProtocol=rt2025.05 | 6.7 | |
| Qwen2.5-7B + cotProtocol=CoT2025.05 | 6.2 | |
| Qwen2.5-3B + rtProtocol=rt2025.05 | 5.3 | |
| Qwen2.5-7B + fs1Protocol=fs12025.05 | 5.3 | |
| Qwen2.5-3B + cotProtocol=CoT2025.05 | 4.8 | |
| Qwen2.5-3B + fs1Protocol=fs12025.05 | 4.5 | |
| Qwen2.5-3B2025.05 | 4.4 | |
| Qwen2.5-1.5B + cotProtocol=CoT2025.05 | 4.1 | |
| Qwen2.5-1.5B + rtProtocol=rt2025.05 | 3.8 | |
| SmolLM2-1.7B + rtProtocol=rt2025.05 | 3.8 | |
| SmolLM2-1.7B + fs1Protocol=fs12025.05 | 3.6 | |
| Qwen2.5-0.5B + fs1Protocol=fs12025.05 | 3.5 | |
| Qwen2.5-1.5B + fs1Protocol=fs12025.05 | 3.5 | |
| SmolLM2-1.7B2025.05 | 3.2 | |
| SmolLM2-1.7B + cotProtocol=CoT2025.05 | 3.2 | |
| Qwen2.5-0.5B + cotProtocol=CoT2025.05 | 3.1 | |
| Qwen2.5-1.5B2025.05 | 3.1 | |
| SmolLM2-360M + rtProtocol=rt2025.05 | 2.9 | |
| SmolLM2-360M + fs1Protocol=fs12025.05 | 2.7 | |
| SmolLM2-360M + cotProtocol=CoT2025.05 | 2.5 | |
| SmolLM2-360M2025.05 | 2.4 | |
| Qwen2.5-0.5B2025.05 | 2.3 | |
| Qwen2.5-0.5B + rtProtocol=rt2025.05 | 2.2 |