Mathematical Reasoning on MATH-500 (Accuracy, Avg., Drop ↓)
95.6AccuracySC@16
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SC@16Model=Deepseek-R1-Distill-Qwen32B2026.03 | 95.6 | — | — | |
| PG@16Model=Deepseek-R1-Distill-Qwen32B2026.03 | 95.6 | — | — | |
| BF16 BaselineModel=Qwen3-4B, W-Bits=BF162026.01 | 95.33 | 70.63 | — | |
| Avg@16Model=Deepseek-R1-Distill-Qwen32B2026.03 | 94.3 | — | — | |
| Reasoning-QATModel=Qwen3-4B, W-Bits=W4A4KV42026.01 | 91.4 | 60.78 | -9.85 | |
| FlatQuantModel=Qwen3-4B, W-Bits=W4A4KV42026.01 | 89.93 | 58.28 | -12.35 | |
| BF16 BaselineModel=R1-1.5B, W-Bits=BF162026.01 | 84.4 | 48.72 | — | |
| PG@16Model=Qwen2.5-7B-Instruct2026.03 | 80.8 | — | — | |
| SC@16Model=Qwen2.5-7B-Instruct2026.03 | 80.2 | — | — | |
| BF16 BaselineModel=Qwen3-0.6B, W-Bits=BF162026.01 | 74 | 41.1 | — | |
| Avg@16Model=Qwen2.5-7B-Instruct2026.03 | 74 | — | — | |
| Reasoning-QATModel=R1-1.5B, W-Bits=W4A4KV42026.01 | 73.2 | 41.31 | -7.41 | |
| FlatQuantModel=R1-1.5B, W-Bits=W4A4KV42026.01 | 64.8 | 38.39 | -10.33 | |
| Reasoning-QATModel=Qwen3-0.6B, W-Bits=W4A4KV42026.01 | 30.27 | 21.44 | -19.66 | |
| FlatQuantModel=Qwen3-0.6B, W-Bits=W4A4KV42026.01 | 21.67 | 17.34 | -23.76 | |
| QuaRotModel=R1-1.5B, W-Bits=W4A4KV42026.01 | 1.2 | 2.11 | -46.61 | |
| QuaRotModel=Qwen3-0.6B, W-Bits=W4A4KV42026.01 | 0 | 4.84 | -36.26 |