Multi-hop Question Answering on HotpotQA (Recall@K and NDCG@K)
48.1Recall@1GRPO
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| GRPOBackbone=Qwen3.5-2B2026.05 | 48.1 | 86.3 | 86.5 | 96.1 | 88.4 | 88.5 | |
| F-GRPOBackbone=Qwen3.5-2B2026.05 | 48.1 | 91.4 | 91.4 | 96.3 | 92.5 | 92.5 | |
| GRPOBackbone=Qwen3-4B2026.05 | 48 | 93 | 93.1 | 96.1 | 93 | 93.1 | |
| F-GRPOBackbone=Qwen3-4B2026.05 | 48 | 92.6 | 92.6 | 96 | 93.3 | 93.3 | |
| RankZephyr2026.05 | 47.4 | 84.9 | 90.3 | 94.8 | 85.9 | 88.6 | |
| MonoT52026.05 | 46.3 | 86.2 | 92.8 | 92.5 | 86.2 | 89.5 | |
| LiT52026.05 | 46.3 | 78.4 | 85.2 | 92.5 | 80.3 | 83.7 | |
| Decoupled SFT (rank. only)Backbone=Qwen3-4B2026.05 | 43.8 | 84.3 | 89.2 | 87.6 | 84 | 86.5 | |
| DuoT52026.05 | 43.2 | 79.8 | 86.2 | 86.4 | 80 | 83.3 | |
| BM252026.05 | 41.8 | 72.2 | 82.7 | 83.6 | 73.2 | 78.5 | |
| SFTBackbone=Qwen3-4B2026.05 | 38.3 | 68.7 | 73 | 76.6 | 69.6 | 71.8 | |
| Decoupled SFT (full)Backbone=Qwen3-4B2026.05 | 32.6 | 49 | 52.9 | 65.1 | 52 | 54 | |
| Zero-shot (single-step)Backbone=Qwen3-4B2026.05 | 30.7 | 61 | 65.4 | 61.4 | 60.2 | 62.4 | |
| Decoupled SFT (rank. only)Backbone=Qwen3.5-2B2026.05 | 21.4 | 51.7 | 68.5 | 42.8 | 47.7 | 56.1 | |
| Decoupled SFT (sel. only)Backbone=Qwen3-4B2026.05 | 9.8 | 19.1 | 20.6 | 19.6 | 18.9 | 19.7 | |
| Decoupled SFT (full)Backbone=Qwen3.5-2B2026.05 | 6 | 16.6 | 25.7 | 11.9 | 14.8 | 19.3 | |
| SFTBackbone=Qwen3.5-2B2026.05 | 5.2 | 15.1 | 25.3 | 10.4 | 13.2 | 18.2 | |
| Random2026.05 | 5 | 15 | 25.1 | 9.9 | 13.1 | 18.1 | |
| Zero-shot (single-step)Backbone=Qwen3.5-2B2026.05 | 4 | 7.4 | 8.4 | 8 | 7.4 | 7.9 | |
| Zero-shot (two-step)Backbone=Qwen3-4B2026.05 | 3 | 5.8 | 6.1 | 6 | 5.8 | 5.9 | |
| Zero-shot (two-step)Backbone=Qwen3.5-2B2026.05 | 1 | 1.9 | 2.1 | 2 | 1.9 | 2 | |
| Decoupled SFT (sel. only)Backbone=Qwen3.5-2B2026.05 | 0.3 | 0.6 | 0.7 | 0.5 | 0.5 | 0.6 |