Multistep Reasoning on MUSR (Accuracy)
61.67AccuracyBase
Evaluation results
| Method | Links | |
|---|---|---|
| BaseBackbone=Qwen3-4B2025.05 | 61.67 | |
| Qwen2.5-14B (Teacher)Model Scale=14B2026.02 | 60.8 | |
| Qwen2.5-14BRole=Teacher2026.02 | 60.8 | |
| Sigmoid capability boundariesFLOPs budget=10^24, Boundary estimation method=no-split 0.98-quantile sigmoid2026.02 | 53.5 | |
| InstructModel Size=32B2025.10 | 50.13 | |
| GIFTModel Size=32B2025.10 | 49.47 | |
| GRPOModel Size=32B2025.10 | 48.68 | |
| GIFTModel Size=7B2025.10 | 46.16 | |
| GRPOModel Size=7B2025.10 | 45.77 | |
| InstructModel Size=7B2025.10 | 42.86 | |
| TSD-KDDistillation Setup=Qwen2.5 (14B -> 1.5B)2026.02 | 39.6 | |
| TSD-KDselection=Adaptive c%2026.02 | 39.4 | |
| Qwen2.5-1.5B (Student)Model Scale=1.5B2026.02 | 38.7 | |
| DistiLLMDistillation Setup=Qwen2.5 (14B -> 1.5B)2026.02 | 38.7 | |
| Qwen2.5-1.5BRole=Student2026.02 | 38.7 | |
| DistiLLM2026.02 | 38.7 | |
| GKDbeta=0.9, Distillation Setup=Qwen2.5 (14B -> 1.5B)2026.02 | 38.5 | |
| GKDbeta=0.92026.02 | 38.5 | |
| MiniLLMDistillation Setup=Qwen2.5 (14B -> 1.5B)2026.02 | 38.3 | |
| MiniLLM2026.02 | 38.3 | |
| Supervised-KDDistillation Setup=Qwen2.5 (14B -> 1.5B)2026.02 | 37.9 | |
| Supervised-KD2026.02 | 37.9 | |
| Speculative KDDistillation Setup=Qwen2.5 (14B -> 1.5B)2026.02 | 37.7 | |
| Speculative KD2026.02 | 37.7 | |
| Sequence-Level KDDistillation Setup=Qwen2.5 (14B -> 1.5B)2026.02 | 36.6 | |
| Sequence-Level KD2026.02 | 36.6 | |
| Vanilla LoRABackbone=Qwen3-4B2025.05 | 32.31 | |
| BaseBackbone=Llama-3.2-3B2025.05 | 30.95 | |
| SHE-LoRABackbone=Qwen3-4B2025.05 | 30.71 | |
| Vanilla LoRABackbone=Llama-3.2-3B2025.05 | 18.05 | |
| SHE-LoRABackbone=Llama-3.2-3B2025.05 | 17.86 |