Multi-needle Retrieval on MRCR
38MRCR 2-Needle SuccessQwen2.5-32B + SPELL
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Qwen2.5-32B + SPELLBackbone=Qwen2.5-32B, Maximum input length=16K, Maximum output length=4K, Alignment=SPELL2025.09 | 38 | 18.5 | 14.7 | 35 | |
| Qwen2.5-32B + RLVRBackbone=Qwen2.5-32B, Maximum input length=16K, Maximum output length=4K, Alignment=RLVR2025.09 | 36.7 | 14.6 | 13 | 25.8 | |
| Qwen2.5-14B + SPELLBackbone=Qwen2.5-14B, Maximum input length=16K, Maximum output length=4K, Alignment=SPELL2025.09 | 35 | 22.1 | 17.8 | 31.6 | |
| Qwen2.5-7B + SPELLBackbone=Qwen2.5-7B, Maximum input length=16K, Maximum output length=4K, Alignment=SPELL2025.09 | 34.5 | 16.5 | 16 | 24.2 | |
| Qwen2.5-32BBackbone=Qwen2.5-32B, Maximum input length=16K, Maximum output length=4K2025.09 | 27 | 11.5 | 12 | 28.1 | |
| Qwen2.5-14BBackbone=Qwen2.5-14B, Maximum input length=16K, Maximum output length=4K2025.09 | 23.1 | 10.5 | 10 | 15.2 | |
| Qwen2.5-7B + RLVRBackbone=Qwen2.5-7B, Maximum input length=16K, Maximum output length=4K, Alignment=RLVR2025.09 | 22 | 12.5 | 10.5 | 18.5 | |
| Qwen2.5-14B + RLVRBackbone=Qwen2.5-14B, Maximum input length=16K, Maximum output length=4K, Alignment=RLVR2025.09 | 20.9 | 9.5 | 9.1 | 25.5 | |
| Qwen2.5-7BBackbone=Qwen2.5-7B, Maximum input length=16K, Maximum output length=4K2025.09 | 6.9 | 2 | 2.2 | 11.5 |