Mathematical Reasoning on GSM8K (test) (ACC, LEN, delta_LEN)
95.24Accuracy (ACC)HAB
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| HABBackbone=Qwen2.5-7B-Instruct2026.05 | 95.24 | 211.1 | — | |
| w thinkingModel=Qwen3-8B2025.10 | 94.62 | 1,491.38 | — | |
| CoDModel=Qwen3-8B2025.10 | 94.4 | 415.8 | 72.12 | |
| CERModel=Qwen3-4B2025.10 | 94.09 | 391.08 | 71 | |
| Vanilla CoT (Zero-Shot)Backbone=Qwen2.5-7B-Instruct2026.05 | 93.94 | 283 | — | |
| CoDModel=Qwen3-4B2025.10 | 93.3 | 385.5 | 71.41 | |
| w thinkingModel=Qwen3-4B2025.10 | 92.8 | 1,348.59 | — | |
| CERModel=Qwen3-8B2025.10 | 92.57 | 668.06 | 55.21 | |
| CCoTModel=Qwen3-8B2025.10 | 92.49 | 739.05 | 50.45 | |
| Laser-DBackbone=DeepSeek-Distill-Qwen-7B2025.09 | 91.6 | 965 | — | |
| Laser-DEBackbone=DeepSeek-Distill-Qwen-7B2025.09 | 91.5 | 948 | — | |
| DAPOBackbone=DeepSeek-Distill-Qwen-7B2025.09 | 90.1 | 583 | — | |
| TokenSkipBackbone=Qwen2.5-7B-Instruct2026.05 | 90.04 | 165.9 | — | |
| AdaThinkBackbone=DeepSeek-Distill-Qwen-7B2025.09 | 88.9 | 304 | — | |
| w/o thinkingModel=Qwen3-8B2025.10 | 88.86 | 272.02 | 79.83 | |
| SFTBackbone=DeepSeek-Distill-Qwen-7B2025.09 | 87.3 | 438 | — | |
| HABBackbone=Llama3.1-8B-Instruct2026.05 | 87.23 | 232.2 | — | |
| BaseBackbone=DeepSeek-Distill-Qwen-7B2025.09 | 87 | 469 | — | |
| w/o thinkingModel=Qwen3-4B2025.10 | 86.5 | 260.96 | 80.65 | |
| DPOBackbone=DeepSeek-Distill-Qwen-7B2025.09 | 86.1 | 438 | — | |
| JETBackbone=DeepSeek-Distill-Qwen-7B2025.09 | 86.1 | 324 | — | |
| LCR1Backbone=DeepSeek-Distill-Qwen-7B2025.09 | 86 | 386 | — | |
| Laser-DBackbone=DeepSeek-Distill-Qwen-1.5B2025.09 | 84.9 | 1,073 | — | |
| O1-PrunerBackbone=Qwen2.5-7B-Instruct2026.05 | 84.63 | 198.7 | — | |
| Laser-DEBackbone=DeepSeek-Distill-Qwen-1.5B2025.09 | 84.1 | 1,179 | — | |
| JETBackbone=DeepSeek-Distill-Qwen-1.5B2025.09 | 83.8 | 605 | — | |
| Vanilla CoT (Zero-Shot)Backbone=Llama3.1-8B-Instruct2026.05 | 83.16 | 244.5 | — | |
| CCoTModel=Qwen3-4B2025.10 | 82.56 | 616.42 | 54.29 | |
| AdaThinkBackbone=DeepSeek-Distill-Qwen-1.5B2025.09 | 82 | 772 | — | |
| SFTBackbone=DeepSeek-Distill-Qwen-1.5B2025.09 | 81.4 | 559 | — | |
| TokenSkipBackbone=Llama3.1-8B-Instruct2026.05 | 81.17 | 124.3 | — | |
| DPOBackbone=DeepSeek-Distill-Qwen-1.5B2025.09 | 80.2 | 530 | — | |
| DAPOBackbone=DeepSeek-Distill-Qwen-1.5B2025.09 | 80 | 826 | — | |
| UDSModel=Qwen-2.5-7B2025.10 | 79.91 | — | — | |
| O1-PrunerBackbone=Llama3.1-8B-Instruct2026.05 | 79.22 | 205.4 | — | |
| GREATSModel=Qwen-2.5-7B2025.10 | 78.61 | — | — | |
| RHO-LossModel=Qwen-2.5-7B2025.10 | 78.38 | — | — | |
| RegularModel=Qwen-2.5-7B2025.10 | 78.23 | — | — | |
| MaxLossModel=Qwen-2.5-7B2025.10 | 77.78 | — | — | |
| RandomModel=Qwen-2.5-7B2025.10 | 77.69 | — | — | |
| MaxGradModel=Qwen-2.5-7B2025.10 | 77.62 | — | — | |
| BaseBackbone=DeepSeek-Distill-Qwen-1.5B2025.09 | 76 | 468 | — | |
| LCR1Backbone=DeepSeek-Distill-Qwen-1.5B2025.09 | 75 | 443 | — | |
| CoDBackbone=Qwen2.5-7B-Instruct2026.05 | 74.24 | 44.2 | — | |
| Sketch-of-ThoughtBackbone=Qwen2.5-7B-Instruct2026.05 | 73.81 | 97.3 | — | |
| Skeleton-of-ThoughtBackbone=Qwen2.5-7B-Instruct2026.05 | 71.65 | 105.1 | — | |
| CoDBackbone=Llama3.1-8B-Instruct2026.05 | 64.29 | 124.3 | — | |
| Sketch-of-ThoughtBackbone=Llama3.1-8B-Instruct2026.05 | 62.55 | 112.8 | — | |
| NAMEx-FullRouting Strategy=Stable-MoE Routing, Fine-tuned on=SmolTalk, Model Size=14B2025.10 | 61.1 | — | — | |
| Skeleton-of-ThoughtBackbone=Llama3.1-8B-Instruct2026.05 | 61.04 | 290.4 | — | |
| NAMEx-FullRouting Strategy=Linear, Fine-tuned on=SmolTalk, Model Size=14B2025.10 | 61 | — | — | |
| NAMEx-FullRouting Strategy=Cosine, Fine-tuned on=SmolTalk, Model Size=14B2025.10 | 60.95 | — | — | |
| EP-CAMExRouting Strategy=Stable-MoE Routing, Fine-tuned on=SmolTalk, Model Size=14B2025.10 | 60.8 | — | — | |
| Qwen1.5-MoERouting Strategy=Stable-MoE Routing, Fine-tuned on=SmolTalk, Model Size=14B2025.10 | 60.65 | — | — | |
| EP-CAMExRouting Strategy=Linear, Fine-tuned on=SmolTalk, Model Size=14B2025.10 | 60.63 | — | — | |
| Qwen1.5-MoERouting Strategy=Linear, Fine-tuned on=SmolTalk, Model Size=14B2025.10 | 60.52 | — | — | |
| EP-CAMExRouting Strategy=Cosine, Fine-tuned on=SmolTalk, Model Size=14B2025.10 | 60.5 | — | — | |
| Qwen1.5-MoERouting Strategy=Cosine, Fine-tuned on=SmolTalk, Model Size=14B2025.10 | 60.28 | — | — | |
| Fisher-IntDim-G + SFTBackbone=Qwen1.5-MoE-A2.7B, MoE compression ratio (p)=50%, Supervised Fine-Tuning (SFT)=true2026.06 | 52 | — | — | |
| Base + SFTBackbone=Qwen1.5-MoE-A2.7B, MoE compression ratio (p)=0%, Supervised Fine-Tuning (SFT)=true2026.06 | 50.8 | — | — | |
| Expert-level Fisher + SFTBackbone=Qwen1.5-MoE-A2.7B, MoE compression ratio (p)=50%, Supervised Fine-Tuning (SFT)=true2026.06 | 46.4 | — | — | |
| MoE comp. (Fisher) + SFTBackbone=Qwen1.5-MoE-A2.7B, MoE compression ratio (p)=50%, Supervised Fine-Tuning (SFT)=true2026.06 | 46 | — | — | |
| MoE-Pruner + SFTBackbone=Qwen1.5-MoE-A2.7B, MoE compression ratio (p)=50%, Supervised Fine-Tuning (SFT)=true2026.06 | 40.6 | — | — | |
| Model SoupBase model=Qwen2.5-1.5B2026.05 | 39.7 | — | — | |
| Single BestBase model=Qwen2.5-1.5B2026.05 | 32.5 | — | — | |
| MoE comp. + SFTBackbone=Qwen1.5-MoE-A2.7B, MoE compression ratio (p)=50%, Supervised Fine-Tuning (SFT)=true2026.06 | 30.3 | — | — | |
| TIESBase model=Qwen2.5-1.5B2026.05 | 30.2 | — | — | |
| MoE comp. (activation) + SFTBackbone=Qwen1.5-MoE-A2.7B, MoE compression ratio (p)=50%, Supervised Fine-Tuning (SFT)=true2026.06 | 28.5 | — | — | |
| PSO-MergingBase model=Qwen2.5-1.5B2026.05 | 26.1 | — | — | |
| Task ArithmeticBase model=Qwen2.5-1.5B2026.05 | 26 | — | — | |
| EvoGMBase model=Qwen2.5-1.5B2026.05 | 24.8 | — | — | |
| Model SwarmBase model=Qwen2.5-1.5B2026.05 | 24.6 | — | — | |
| DAREBase model=Qwen2.5-1.5B2026.05 | 24.3 | — | — | |
| CMABase model=Qwen2.5-1.5B2026.05 | 21.1 | — | — | |
| MTLBase model=Qwen2.5-1.5B2026.05 | 18.8 | — | — | |
| BaseBase model=Qwen2.5-1.5B2026.05 | 15.5 | — | — |