Multi-hop Question Answering on MuSiQue in-distribution
56.91EMCRAFT7B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| CRAFT7BType=Trained, CRAFT version=v4, Training Mode=RL2026.02 | 56.91 | 66.47 | 86.7 | |
| Deepseek-V3.2Type=API, CRAFT version=v42026.02 | 53.2 | 67.96 | 88.25 | |
| GPT-5-miniType=API, CRAFT version=v42026.02 | 52.05 | 68.1 | 93.15 | |
| Gemini-2.5-FlashType=API, CRAFT version=v42026.02 | 51.25 | 62.92 | 90.2 | |
| Qwen3-30BType=Pre-Trained, CRAFT version=v42026.02 | 46.75 | 56.34 | 84.95 | |
| Qwen3-4BType=Pre-Trained, CRAFT version=v42026.02 | 38.5 | 47.61 | 77.55 | |
| Qwen2.5-7BType=Pre-Trained, CRAFT version=v42026.02 | 38.35 | 49.29 | 76.6 | |
| CRAFT3BType=Trained, CRAFT version=v4, Training Mode=RL2026.02 | 32.71 | 43.14 | 71.4 | |
| Qwen2.5-7BType=Trained, CRAFT version=v4, Training Mode=SFT2026.02 | 31.8 | 42.71 | 66.6 | |
| Qwen2.5-3BType=Trained, CRAFT version=v4, Training Mode=SFT2026.02 | 23.6 | 33.1 | 50.15 | |
| Qwen2.5-3BType=Pre-Trained, CRAFT version=v42026.02 | 21.95 | 32.3 | 41.15 | |
| CRAFT1.5BType=Trained, CRAFT version=v4, Training Mode=RL2026.02 | 16.41 | 23.27 | 29.57 | |
| CRAFT0.5BType=Trained, CRAFT version=v4, Training Mode=RL2026.02 | 8.45 | 15.11 | 25.4 | |
| Qwen2.5-1.5BType=Trained, CRAFT version=v4, Training Mode=SFT2026.02 | 7.25 | 12.85 | 16.4 | |
| Qwen2.5-1.5BType=Pre-Trained, CRAFT version=v42026.02 | 4.85 | 9.78 | 16.65 | |
| Qwen2.5-0.5BType=Trained, CRAFT version=v4, Training Mode=SFT2026.02 | 0.5 | 4.62 | 13.5 | |
| Qwen2.5-0.5BType=Pre-Trained, CRAFT version=v42026.02 | 0.35 | 2.69 | 4.85 |