Multi-hop Question Answering on Mintaka
77.4Pass@1o3-mini
Evaluation results
| Method | Links | |
|---|---|---|
| o3-mini2025.05 | 77.4 | |
| R1-70B2025.05 | 75.5 | |
| Qwen2.5-72B2025.05 | 73.6 | |
| QwQ-32B2025.05 | 70.8 | |
| Qwen2.5-32B + cotProtocol=CoT2025.05 | 69.6 | |
| Qwen2.5-32B + fs1Protocol=fs12025.05 | 68.2 | |
| Qwen2.5-32B + rtProtocol=rt2025.05 | 68 | |
| Qwen2.5-14B + rtProtocol=rt2025.05 | 67.8 | |
| Qwen2.5-32B2025.05 | 67.4 | |
| Qwen2.5-14B + cotProtocol=CoT2025.05 | 66.4 | |
| Qwen2.5-14B + fs1Protocol=fs12025.05 | 65.4 | |
| Qwen2.5-14B2025.05 | 62.4 | |
| Qwen2.5-7B + cotProtocol=CoT2025.05 | 58 | |
| Qwen2.5-7B + rtProtocol=rt2025.05 | 57.6 | |
| Qwen2.5-7B + fs1Protocol=fs12025.05 | 55.1 | |
| Qwen2.5-7B2025.05 | 54.8 | |
| Qwen2.5-3B + rtProtocol=rt2025.05 | 49.5 | |
| Qwen2.5-3B + fs1Protocol=fs12025.05 | 44.4 | |
| Qwen2.5-3B + cotProtocol=CoT2025.05 | 43.1 | |
| Qwen2.5-3B2025.05 | 39.6 | |
| SmolLM2-1.7B + rtProtocol=rt2025.05 | 36.6 | |
| SmolLM2-1.7B + fs1Protocol=fs12025.05 | 34.1 | |
| Qwen2.5-1.5B + cotProtocol=CoT2025.05 | 31.8 | |
| Qwen2.5-1.5B2025.05 | 31.6 | |
| Qwen2.5-1.5B + fs1Protocol=fs12025.05 | 30.1 | |
| SmolLM2-1.7B + cotProtocol=CoT2025.05 | 29.5 | |
| Qwen2.5-1.5B + rtProtocol=rt2025.05 | 29.4 | |
| SmolLM2-1.7B2025.05 | 29.3 | |
| Qwen2.5-0.5B + cotProtocol=CoT2025.05 | 21.4 | |
| SmolLM2-360M + rtProtocol=rt2025.05 | 20.2 | |
| Qwen2.5-0.5B + fs1Protocol=fs12025.05 | 20.2 | |
| SmolLM2-360M + fs1Protocol=fs12025.05 | 19.7 | |
| SmolLM2-360M + cotProtocol=CoT2025.05 | 18.8 | |
| Qwen2.5-0.5B + rtProtocol=rt2025.05 | 17.8 | |
| SmolLM2-360M2025.05 | 17.5 | |
| Qwen2.5-0.5B2025.05 | 13.1 |