Multi-hop Question Answering on CWQ
55.8Pass@1o3-mini
Evaluation Results
| Method | Links | |
|---|---|---|
| o3-miniType=Baseline2025.05 | 55.8 | |
| R1-70BType=Baseline, Size=70B2025.05 | 50.1 | |
| Qwen2.5-72BType=Baseline, Size=72B2025.05 | 48.1 | |
| QwQ-32BType=Baseline, Size=32B2025.05 | 47.9 | |
| Qwen2.5-32B + fs1Backbone=Qwen2.5-32B, Protocol=fs1 fine-tuned2025.05 | 47.7 | |
| Qwen2.5-32B + rtBackbone=Qwen2.5-32B, Protocol=rt fine-tuned2025.05 | 47.1 | |
| Qwen2.5-14B + fs1Backbone=Qwen2.5-14B, Protocol=fs1 fine-tuned2025.05 | 45.4 | |
| Qwen2.5-14B + rtBackbone=Qwen2.5-14B, Protocol=rt fine-tuned2025.05 | 45.1 | |
| Qwen2.5-32B + cotBackbone=Qwen2.5-32B, Protocol=CoT fine-tuned2025.05 | 43.5 | |
| Qwen2.5-32BBackbone=Qwen2.5-32B, Protocol=Instruction-tuned2025.05 | 42.8 | |
| Qwen2.5-14B + cotBackbone=Qwen2.5-14B, Protocol=CoT fine-tuned2025.05 | 42.2 | |
| Qwen2.5-7B + fs1Backbone=Qwen2.5-7B, Protocol=fs1 fine-tuned2025.05 | 40.8 | |
| Qwen2.5-7B + rtBackbone=Qwen2.5-7B, Protocol=rt fine-tuned2025.05 | 40.1 | |
| Qwen2.5-14BBackbone=Qwen2.5-14B, Protocol=Instruction-tuned2025.05 | 39.2 | |
| Qwen2.5-7B + cotBackbone=Qwen2.5-7B, Protocol=CoT fine-tuned2025.05 | 38.3 | |
| Qwen2.5-7BBackbone=Qwen2.5-7B, Protocol=Instruction-tuned2025.05 | 37.6 | |
| Qwen2.5-3B + rtBackbone=Qwen2.5-3B, Protocol=rt fine-tuned2025.05 | 36.3 | |
| Qwen2.5-3B + fs1Backbone=Qwen2.5-3B, Protocol=fs1 fine-tuned2025.05 | 33 | |
| Qwen2.5-3BBackbone=Qwen2.5-3B, Protocol=Instruction-tuned2025.05 | 31.7 | |
| SmolLM2-1.7B + rtBackbone=SmolLM2-1.7B, Protocol=rt fine-tuned2025.05 | 30.6 | |
| SmolLM2-1.7B + fs1Backbone=SmolLM2-1.7B, Protocol=fs1 fine-tuned2025.05 | 30.5 | |
| Qwen2.5-3B + cotBackbone=Qwen2.5-3B, Protocol=CoT fine-tuned2025.05 | 30.2 | |
| SmolLM2-1.7B + cotBackbone=SmolLM2-1.7B, Protocol=CoT fine-tuned2025.05 | 28.5 | |
| Qwen2.5-1.5B + fs1Backbone=Qwen2.5-1.5B, Protocol=fs1 fine-tuned2025.05 | 26.3 | |
| Qwen2.5-1.5B + rtBackbone=Qwen2.5-1.5B, Protocol=rt fine-tuned2025.05 | 25.5 | |
| Qwen2.5-1.5B + cotBackbone=Qwen2.5-1.5B, Protocol=CoT fine-tuned2025.05 | 25.2 | |
| SmolLM2-1.7BBackbone=SmolLM2-1.7B, Protocol=Instruction-tuned2025.05 | 24.8 | |
| Qwen2.5-1.5BBackbone=Qwen2.5-1.5B, Protocol=Instruction-tuned2025.05 | 23.4 | |
| Qwen2.5-0.5B + fs1Backbone=Qwen2.5-0.5B, Protocol=fs1 fine-tuned2025.05 | 20.9 | |
| SmolLM2-360M + rtBackbone=SmolLM2-360M, Protocol=rt fine-tuned2025.05 | 19.2 | |
| Qwen2.5-0.5B + rtBackbone=Qwen2.5-0.5B, Protocol=rt fine-tuned2025.05 | 19 | |
| SmolLM2-360M + fs1Backbone=SmolLM2-360M, Protocol=fs1 fine-tuned2025.05 | 17.9 | |
| Qwen2.5-0.5B + cotBackbone=Qwen2.5-0.5B, Protocol=CoT fine-tuned2025.05 | 16.1 | |
| SmolLM2-360M + cotBackbone=SmolLM2-360M, Protocol=CoT fine-tuned2025.05 | 15.1 | |
| SmolLM2-360MBackbone=SmolLM2-360M, Protocol=Instruction-tuned2025.05 | 14.8 | |
| Qwen2.5-0.5BBackbone=Qwen2.5-0.5B, Protocol=Instruction-tuned2025.05 | 13.5 |