Mathematical Reasoning on MATH 500 (Accuracy and Token Cost)
92.1AccuracyVanilla
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| VanillaModel=DS-7B2026.01 | 92.1 | 4,097 | — | — | — | |
| EntrocutModel=DS-7B2026.01 | 91.1 | 3,046 | 2.7 | 2.4 | — | |
| TIPModel=DS-7B2026.01 | 90.5 | 3,055 | 1.7 | 1.3 | — | |
| DEERModel=DS-7B2026.01 | 89 | 2,342 | 1.5 | 1.5 | — | |
| NOWAITModel=DS-7B2026.01 | 88.7 | 2,768 | — | — | — | |
| TACLerMode=NoThinking2026.01 | 88.2 | — | — | — | 2,532 | |
| Poisoned-RM (Thought-Transfer)Objective=Advertisement Injection2026.01 | 88.2 | — | — | — | — | |
| Poisoned-RM (Thought-Transfer)Objective=Concept Manipulation2026.01 | 88.2 | — | — | — | — | |
| CRTbase_model=DeepSeek-1.5B2026.02 | 86.06 | — | — | — | 3,826 | |
| AdaptThinkMode=Efficient Reasoning2026.01 | 86 | — | — | — | 2,511 | |
| O1-Pruner-1.0base_model=DeepSeek-1.5B2026.02 | 85.75 | — | — | — | 4,174 | |
| ThinkPrune-1kbase_model=DeepSeek-1.5B2026.02 | 85.69 | — | — | — | 4,206 | |
| L1base_model=DeepSeek-1.5B2026.02 | 85.69 | — | — | — | 4,891 | |
| O1-Pruner-4.0base_model=DeepSeek-1.5B2026.02 | 85.56 | — | — | — | 4,452 | |
| ACPObase_model=DeepSeek-1.5B2026.02 | 85.44 | — | — | — | 4,175 | |
| ThinkPrune-500base_model=DeepSeek-1.5B2026.02 | 85.25 | — | — | — | 4,455 | |
| DeepSeek-1.5B2026.02 | 85.19 | — | — | — | 4,987 | |
| TLMREMode=Efficient Reasoning2026.01 | 85 | — | — | — | 3,007 | |
| AutoThinkMode=Efficient Reasoning2026.01 | 83.8 | — | — | — | 2,128 | |
| VanillaModel=DS-1.5B2026.01 | 83.1 | 4,984 | — | — | — | |
| DASTMode=Efficient Reasoning2026.01 | 83 | — | — | — | 2,428 | |
| ShorterBetterbase_model=DeepSeek-1.5B2026.02 | 82.47 | — | — | — | 3,205 | |
| O1-PrunerMode=Efficient Reasoning2026.01 | 82.2 | — | — | — | 3,212 | |
| EntrocutModel=DS-1.5B2026.01 | 81.6 | 3,341 | 2.4 | 12.4 | — | |
| R1-QwenMode=Thinking2026.01 | 81.2 | — | — | — | 4,856 | |
| OverThinkMode=Efficient Reasoning2026.01 | 81.2 | — | — | — | 4,131 | |
| TIPModel=DS-1.5B2026.01 | 80.8 | 3,419 | 1.5 | 2.9 | — | |
| NOWAITModel=DS-1.5B2026.01 | 79 | 3,086 | — | — | — | |
| Qwen-14B (Base Model)Objective=Advertisement Injection2026.01 | 78 | — | — | — | — | |
| Qwen-14B (Base Model)Objective=Concept Manipulation2026.01 | 78 | — | — | — | — | |
| DEERModel=DS-1.5B2026.01 | 70.1 | 2,575 | 0.4 | 1.7 | — | |
| R1-QwenMode=NoThinking2026.01 | 67.8 | — | — | — | 1,069 | |
| ModelMergingMode=Efficient Reasoning2026.01 | 63 | — | — | — | 2,723 | |
| Info-Gain SamplerK=1, Backbone=Dream-7B, Block Size=16, Attention Type=full-attention2026.02 | 51.3 | — | — | — | — | |
| KLASSK=1, Backbone=Dream-7B, Block Size=16, Attention Type=full-attention2026.02 | 47.2 | — | — | — | — | |
| EntropyK=1, Backbone=Dream-7B, Block Size=16, Attention Type=full-attention2026.02 | 47 | — | — | — | — | |
| PC-SamplerK=1, Backbone=Dream-7B, Block Size=16, Attention Type=full-attention2026.02 | 46.8 | — | — | — | — | |
| HyPERModel=OLMoE2026.02 | 46.7 | 0.73 | — | — | — | |
| ConfidenceK=1, Backbone=Dream-7B, Block Size=16, Attention Type=full-attention2026.02 | 46.6 | — | — | — | — | |
| MarginK=1, Backbone=Dream-7B, Block Size=16, Attention Type=full-attention2026.02 | 46.5 | — | — | — | — | |
| DeepConfModel=OLMoE2026.02 | 43.7 | 0.55 | — | — | — | |
| HyPERModel=DeepSeek-V22026.02 | 40.8 | 0.94 | — | — | — | |
| DeepConfModel=DeepSeek-V22026.02 | 40.2 | 0.65 | — | — | — | |
| Self-CertaintyModel=DeepSeek-V22026.02 | 38.3 | 0.75 | — | — | — | |
| RoEModel=OLMoE2026.02 | 38.2 | 2.35 | — | — | — | |
| Self-CertaintyModel=OLMoE2026.02 | 37.9 | 1.09 | — | — | — | |
| SCModel=DeepSeek-V22026.02 | 37.8 | 1 | — | — | — | |
| RoEModel=DeepSeek-V22026.02 | 37.5 | 1.93 | — | — | — | |
| SCModel=OLMoE2026.02 | 37.3 | 1 | — | — | — | |
| Info-Gain SamplerK=2, Backbone=Dream-7B, Block Size=16, Attention Type=full-attention2026.02 | 34.2 | — | — | — | — | |
| PC-SamplerK=2, Backbone=Dream-7B, Block Size=16, Attention Type=full-attention2026.02 | 32.3 | — | — | — | — | |
| KLASSK=2, Backbone=Dream-7B, Block Size=16, Attention Type=full-attention2026.02 | 30.4 | — | — | — | — | |
| ConfidenceK=2, Backbone=Dream-7B, Block Size=16, Attention Type=full-attention2026.02 | 29.4 | — | — | — | — | |
| MarginK=2, Backbone=Dream-7B, Block Size=16, Attention Type=full-attention2026.02 | 28.8 | — | — | — | — | |
| EntropyK=2, Backbone=Dream-7B, Block Size=16, Attention Type=full-attention2026.02 | 27 | — | — | — | — | |
| UniformK=1, Backbone=Dream-7B, Block Size=16, Attention Type=full-attention2026.02 | 15.8 | — | — | — | — | |
| UniformK=2, Backbone=Dream-7B, Block Size=16, Attention Type=full-attention2026.02 | 14.3 | — | — | — | — |