Mathematical Reasoning on MATH 500 (Accuracy, Token Usage, Compression Ratio)
95.2AccuracyWHISPER
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| WHISPERBase Model=Qwen3-14B2025.10 | 95.2 | 2,176 | 49.5 | |
| ChainofDraftBase Model=Qwen3-14B2025.10 | 94.8 | 3,201 | 72.8 | |
| OriginalBase Model=Qwen3-14B2025.10 | 94.5 | 4,398 | 100 | |
| BeConciseBase Model=Qwen3-14B2025.10 | 94.5 | 3,682 | 83.7 | |
| GEPABase Model=Qwen3-14B2025.10 | 94.3 | 2,993 | 68.1 | |
| DEER*Base Model=Qwen3-14B2025.10 | 93.9 | 3,067 | 69.7 | |
| BeConciseBase Model=DeepSeek-R1-Distill-Qwen-14B2025.10 | 93.5 | 3,331 | 92.4 | |
| OriginalBase Model=DeepSeek-R1-Distill-Qwen-14B2025.10 | 92.9 | 3,605 | 100 | |
| WHISPERBase Model=DeepSeek-R1-Distill-Qwen-14B2025.10 | 92.5 | 2,521 | 69.9 | |
| GEPABase Model=DeepSeek-R1-Distill-Qwen-14B2025.10 | 92.1 | 2,763 | 76.6 | |
| DEER*Base Model=DeepSeek-R1-Distill-Qwen-14B2025.10 | 91.2 | 2,588 | 71.8 | |
| BeConciseBase Model=DeepSeek-R1-Distill-LLaMA-8B2025.10 | 89.5 | 3,753 | 91.5 | |
| WHISPERBase Model=DeepSeek-R1-Distill-LLaMA-8B2025.10 | 89.3 | 2,823 | 68.8 | |
| OriginalBase Model=DeepSeek-R1-Distill-LLaMA-8B2025.10 | 89.2 | 4,102 | 100 | |
| ChainofDraftBase Model=DeepSeek-R1-Distill-LLaMA-8B2025.10 | 89.2 | 3,557 | 86.7 | |
| ChainofDraftBase Model=DeepSeek-R1-Distill-Qwen-14B2025.10 | 88.9 | 2,433 | 67.5 | |
| GEPABase Model=DeepSeek-R1-Distill-LLaMA-8B2025.10 | 88.6 | 3,446 | 84 | |
| NoThinkingBase Model=Qwen3-14B2025.10 | 86.3 | 900 | 20.5 | |
| DEER*Base Model=DeepSeek-R1-Distill-LLaMA-8B2025.10 | 86.2 | 3,021 | 73.6 | |
| NoThinkingBase Model=DeepSeek-R1-Distill-Qwen-14B2025.10 | 77.7 | 730 | 20.2 | |
| NoThinkingBase Model=DeepSeek-R1-Distill-LLaMA-8B2025.10 | 69.9 | 972 | 23.7 |