Multi-hop Question Answering on ExaQT
49.7Pass@1o3-mini
Evaluation Results
| Method | Links | |
|---|---|---|
| o3-mini2025.05 | 49.7 | |
| R1-70BSize=70B2025.05 | 47.6 | |
| Qwen2.5-72BSize=72B2025.05 | 44 | |
| QwQ-32BSize=32B2025.05 | 39 | |
| Qwen2.5-32B + cotBackbone=Qwen2.5-32B, Protocol=CoT2025.05 | 36.6 | |
| Qwen2.5-32B + rtBackbone=Qwen2.5-32B, Protocol=rt2025.05 | 36.6 | |
| Qwen2.5-32BBackbone=Qwen2.5-32B2025.05 | 36.2 | |
| Qwen2.5-32B + fs1Backbone=Qwen2.5-32B, Protocol=fs12025.05 | 36.1 | |
| Qwen2.5-14B + cotBackbone=Qwen2.5-14B, Protocol=CoT2025.05 | 35.6 | |
| Qwen2.5-14B + rtBackbone=Qwen2.5-14B, Protocol=rt2025.05 | 35.2 | |
| Qwen2.5-14B + fs1Backbone=Qwen2.5-14B, Protocol=fs12025.05 | 33.9 | |
| Qwen2.5-14BBackbone=Qwen2.5-14B2025.05 | 33.6 | |
| Qwen2.5-7B + rtBackbone=Qwen2.5-7B, Protocol=rt2025.05 | 29.6 | |
| Qwen2.5-7B + cotBackbone=Qwen2.5-7B, Protocol=CoT2025.05 | 29.2 | |
| Qwen2.5-7BBackbone=Qwen2.5-7B2025.05 | 28.1 | |
| Qwen2.5-7B + fs1Backbone=Qwen2.5-7B, Protocol=fs12025.05 | 27.2 | |
| Qwen2.5-3B + rtBackbone=Qwen2.5-3B, Protocol=rt2025.05 | 23.5 | |
| Qwen2.5-3B + cotBackbone=Qwen2.5-3B, Protocol=CoT2025.05 | 22.2 | |
| Qwen2.5-3BBackbone=Qwen2.5-3B2025.05 | 21.4 | |
| Qwen2.5-3B + fs1Backbone=Qwen2.5-3B, Protocol=fs12025.05 | 20.5 | |
| SmolLM2-1.7B + rtBackbone=SmolLM2-1.7B, Protocol=rt2025.05 | 18.4 | |
| Qwen2.5-1.5B + cotBackbone=Qwen2.5-1.5B, Protocol=CoT2025.05 | 17.9 | |
| SmolLM2-1.7B + fs1Backbone=SmolLM2-1.7B, Protocol=fs12025.05 | 17.9 | |
| SmolLM2-1.7B + cotBackbone=SmolLM2-1.7B, Protocol=CoT2025.05 | 17.7 | |
| SmolLM2-1.7BBackbone=SmolLM2-1.7B2025.05 | 17.6 | |
| Qwen2.5-1.5B + rtBackbone=Qwen2.5-1.5B, Protocol=rt2025.05 | 17.3 | |
| Qwen2.5-1.5BBackbone=Qwen2.5-1.5B2025.05 | 17 | |
| Qwen2.5-1.5B + fs1Backbone=Qwen2.5-1.5B, Protocol=fs12025.05 | 16.2 | |
| SmolLM2-360M + rtBackbone=SmolLM2-360M, Protocol=rt2025.05 | 11.1 | |
| Qwen2.5-0.5B + cotBackbone=Qwen2.5-0.5B, Protocol=CoT2025.05 | 10.4 | |
| SmolLM2-360M + cotBackbone=SmolLM2-360M, Protocol=CoT2025.05 | 10.1 | |
| Qwen2.5-0.5B + fs1Backbone=Qwen2.5-0.5B, Protocol=fs12025.05 | 10.1 | |
| SmolLM2-360M + fs1Backbone=SmolLM2-360M, Protocol=fs12025.05 | 9.3 | |
| Qwen2.5-0.5B + rtBackbone=Qwen2.5-0.5B, Protocol=rt2025.05 | 8.9 | |
| SmolLM2-360MBackbone=SmolLM2-360M2025.05 | 8.8 | |
| Qwen2.5-0.5BBackbone=Qwen2.5-0.5B2025.05 | 5.8 |