List your company
ResearchFormerly WizWand

Multi-hop Question Answering on CWQ

55.8Pass@1o3-mini
11.80823.22934.6546.071May 16, 2025
Updated 6mo ago

Evaluation results

MethodLinks
o3-miniType=Baseline2025.0555.8
R1-70BType=Baseline, Size=70B2025.0550.1
Qwen2.5-72BType=Baseline, Size=72B2025.0548.1
QwQ-32BType=Baseline, Size=32B2025.0547.9
Qwen2.5-32B + fs1Backbone=Qwen2.5-32B, Protocol=fs1 fine-tuned2025.0547.7
Qwen2.5-32B + rtBackbone=Qwen2.5-32B, Protocol=rt fine-tuned2025.0547.1
Qwen2.5-14B + fs1Backbone=Qwen2.5-14B, Protocol=fs1 fine-tuned2025.0545.4
Qwen2.5-14B + rtBackbone=Qwen2.5-14B, Protocol=rt fine-tuned2025.0545.1
Qwen2.5-32B + cotBackbone=Qwen2.5-32B, Protocol=CoT fine-tuned2025.0543.5
Qwen2.5-32BBackbone=Qwen2.5-32B, Protocol=Instruction-tuned2025.0542.8
Qwen2.5-14B + cotBackbone=Qwen2.5-14B, Protocol=CoT fine-tuned2025.0542.2
Qwen2.5-7B + fs1Backbone=Qwen2.5-7B, Protocol=fs1 fine-tuned2025.0540.8
Qwen2.5-7B + rtBackbone=Qwen2.5-7B, Protocol=rt fine-tuned2025.0540.1
Qwen2.5-14BBackbone=Qwen2.5-14B, Protocol=Instruction-tuned2025.0539.2
Qwen2.5-7B + cotBackbone=Qwen2.5-7B, Protocol=CoT fine-tuned2025.0538.3
Qwen2.5-7BBackbone=Qwen2.5-7B, Protocol=Instruction-tuned2025.0537.6
Qwen2.5-3B + rtBackbone=Qwen2.5-3B, Protocol=rt fine-tuned2025.0536.3
Qwen2.5-3B + fs1Backbone=Qwen2.5-3B, Protocol=fs1 fine-tuned2025.0533
Qwen2.5-3BBackbone=Qwen2.5-3B, Protocol=Instruction-tuned2025.0531.7
SmolLM2-1.7B + rtBackbone=SmolLM2-1.7B, Protocol=rt fine-tuned2025.0530.6
SmolLM2-1.7B + fs1Backbone=SmolLM2-1.7B, Protocol=fs1 fine-tuned2025.0530.5
Qwen2.5-3B + cotBackbone=Qwen2.5-3B, Protocol=CoT fine-tuned2025.0530.2
SmolLM2-1.7B + cotBackbone=SmolLM2-1.7B, Protocol=CoT fine-tuned2025.0528.5
Qwen2.5-1.5B + fs1Backbone=Qwen2.5-1.5B, Protocol=fs1 fine-tuned2025.0526.3
Qwen2.5-1.5B + rtBackbone=Qwen2.5-1.5B, Protocol=rt fine-tuned2025.0525.5
Qwen2.5-1.5B + cotBackbone=Qwen2.5-1.5B, Protocol=CoT fine-tuned2025.0525.2
SmolLM2-1.7BBackbone=SmolLM2-1.7B, Protocol=Instruction-tuned2025.0524.8
Qwen2.5-1.5BBackbone=Qwen2.5-1.5B, Protocol=Instruction-tuned2025.0523.4
Qwen2.5-0.5B + fs1Backbone=Qwen2.5-0.5B, Protocol=fs1 fine-tuned2025.0520.9
SmolLM2-360M + rtBackbone=SmolLM2-360M, Protocol=rt fine-tuned2025.0519.2
Qwen2.5-0.5B + rtBackbone=Qwen2.5-0.5B, Protocol=rt fine-tuned2025.0519
SmolLM2-360M + fs1Backbone=SmolLM2-360M, Protocol=fs1 fine-tuned2025.0517.9
Qwen2.5-0.5B + cotBackbone=Qwen2.5-0.5B, Protocol=CoT fine-tuned2025.0516.1
SmolLM2-360M + cotBackbone=SmolLM2-360M, Protocol=CoT fine-tuned2025.0515.1
SmolLM2-360MBackbone=SmolLM2-360M, Protocol=Instruction-tuned2025.0514.8
Qwen2.5-0.5BBackbone=Qwen2.5-0.5B, Protocol=Instruction-tuned2025.0513.5