Mathematical Reasoning on GSM8K (Accuracy, Avg., Drop ↓)
94.49AccuracyBF16 Baseline
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BF16 BaselineModel=Qwen3-4B, W-Bits=BF162026.01 | 94.49 | 70.63 | — | |
| Reasoning-QATModel=Qwen3-4B, W-Bits=W4A4KV42026.01 | 92.42 | 60.78 | -9.85 | |
| FlatQuantModel=Qwen3-4B, W-Bits=W4A4KV42026.01 | 92.12 | 58.28 | -12.35 | |
| BF16 BaselineModel=R1-1.5B, W-Bits=BF162026.01 | 84.61 | 48.72 | — | |
| BF16 BaselineModel=Qwen3-0.6B, W-Bits=BF162026.01 | 79 | 41.1 | — | |
| FlatQuantModel=R1-1.5B, W-Bits=W4A4KV42026.01 | 78.62 | 38.39 | -10.33 | |
| Reasoning-QATModel=R1-1.5B, W-Bits=W4A4KV42026.01 | 77.94 | 41.31 | -7.41 | |
| Reasoning-QATModel=Qwen3-0.6B, W-Bits=W4A4KV42026.01 | 48.62 | 21.44 | -19.66 | |
| FlatQuantModel=Qwen3-0.6B, W-Bits=W4A4KV42026.01 | 33.06 | 17.34 | -23.76 | |
| QuaRotModel=R1-1.5B, W-Bits=W4A4KV42026.01 | 0.76 | 2.11 | -46.61 | |
| QuaRotModel=Qwen3-0.6B, W-Bits=W4A4KV42026.01 | 0 | 4.84 | -36.26 |