Mathematical Reasoning on MATH500 (Acc, Len, TE)
93.5AccuracySTACK
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| STACKBackbone=DeepSeek-R1-Distill-Qwen-7B2026.04 | 93.5 | 1,733 | 5.39 | |
| TokenSqueezeBackbone=DeepSeek-R1-Distill-Qwen-7B2026.04 | 92.8 | 1,957 | 4.74 | |
| MuTISBackbone=DeepSeek-R1-Distill-Qwen-7B2026.04 | 92.3 | 1,887 | 4.89 | |
| PromptBackbone=DeepSeek-R1-Distill-Qwen-7B2026.04 | 91.7 | 3,654 | 2.51 | |
| OriginalBackbone=DeepSeek-R1-Distill-Qwen-7B2026.04 | 91.2 | 4,010 | 2.27 | |
| ConCISEBackbone=DeepSeek-R1-Distill-Qwen-7B2026.04 | 91 | 2,016 | 4.51 | |
| STACKBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.04 | 85.3 | 1,783 | 4.78 | |
| FP16Backbone=Phi-4-14B, K/V=16/16, bits/elem=16.0, UP (Uniform Precision)=true2026.06 | 84.9 | 2,839 | — | |
| KVarNBackbone=Phi-4-14B, K/V=2/2, bits/elem=2.3, UP (Uniform Precision)=true2026.06 | 84.8 | 2,984 | — | |
| MuTISBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.04 | 84.6 | 1,794 | 4.72 | |
| PromptBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.04 | 83.7 | 4,313 | 1.94 | |
| ConCISEBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.04 | 83.3 | 2,547 | 3.27 | |
| TokenSqueezeBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.04 | 83.2 | 2,447 | 3.4 | |
| FP16Backbone=Qwen3-4B, K/V=16/16, bits/elem=16.0, UP (Uniform Precision)=true2026.06 | 82.6 | 3,857 | — | |
| OriginalBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.04 | 82.4 | 4,876 | 1.69 | |
| KVarNBackbone=Qwen3-4B, K/V=2/2, bits/elem=2.3, UP (Uniform Precision)=true2026.06 | 79.2 | 3,925 | — | |
| QuaRotBackbone=Qwen3-4B, K/V=2/2, bits/elem=2.3, UP (Uniform Precision)=true2026.06 | 78.9 | 3,907 | — | |
| KittyBackbone=Qwen3-4B, K/V=2/2, bits/elem=2.4, UP (Uniform Precision)=false2026.06 | 78.5 | 3,834 | — | |
| KIVIBackbone=Qwen3-4B, K/V=2/2, bits/elem=2.3, UP (Uniform Precision)=true2026.06 | 77.8 | 3,957 | — | |
| TurboQuantBackbone=Qwen3-4B, K/V=3/3, bits/elem=4.6, UP (Uniform Precision)=true, Implementation=vLLM2026.06 | 77 | 3,917 | — | |
| QuaRotBackbone=Phi-4-14B, K/V=2/2, bits/elem=2.3, UP (Uniform Precision)=true2026.06 | 77 | 3,029 | — | |
| PolarQuantBackbone=Phi-4-14B, K/V=4/2, bits/elem=3.3, UP (Uniform Precision)=true2026.06 | 75.8 | 3,023 | — | |
| KittyBackbone=Phi-4-14B, K/V=2/2, bits/elem=2.4, UP (Uniform Precision)=false2026.06 | 74.5 | 3,171 | — | |
| KIVIBackbone=Phi-4-14B, K/V=2/2, bits/elem=2.3, UP (Uniform Precision)=true2026.06 | 74.4 | 3,180 | — | |
| TurboQuantBackbone=Phi-4-14B, K/V=3/3, bits/elem=4.5, UP (Uniform Precision)=true, Implementation=vLLM2026.06 | 74.2 | 3,250 | — | |
| KVQuant-1%Backbone=Phi-4-14B, K/V=2/2, bits/elem=2.4, UP (Uniform Precision)=false2026.06 | 72.3 | 3,588 | — | |
| PolarQuantBackbone=Qwen3-4B, K/V=4/2, bits/elem=3.3, UP (Uniform Precision)=true2026.06 | 71.1 | 5,259 | — | |
| KVQuant-1%Backbone=Qwen3-4B, K/V=2/2, bits/elem=2.4, UP (Uniform Precision)=false2026.06 | 67.5 | 5,021 | — |