Mathematical Reasoning on GSM8k (Accuracy, Delta)
78.09AccuracyDense
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DenseModel=Llama-3.1-8B-Instruct2026.03 | 78.09 | — | |
| DenseModel=Llama-3.1-8B-Instruct, Compression Method=Dense2026.03 | 78.09 | — | |
| DenseModel=Llama-3.1-8B-Instruct2026.03 | 78.09 | — | |
| ZipCalModel=Llama-3.1-8B-Instruct, Calibration Category=Mean2026.03 | 76.25 | 0.43 | |
| COLAModel=Llama-3.1-8B-Instruct, Calibration Category=Mean2026.03 | 75.82 | — | |
| ZipCalModel=Llama-3.1-8B-Instruct, Compression Method=Wanda, Compression Setting (Sparsity/Bit-width)=25%2026.03 | 75.8 | -0.02 | |
| DenseModel=gemma-2-9b-it, Compression Method=Dense2026.03 | 75.44 | — | |
| DenseModel=gemma-2-9b-it2026.03 | 75.44 | — | |
| ZipCalModel=gemma-2-9b-it, Compression Method=Wanda, Compression Setting (Sparsity/Bit-width)=25%2026.03 | 74.75 | 0.44 | |
| ZipCalModel=gemma-2-9b-it, Calibration Category=Mean2026.03 | 74.49 | 0.18 | |
| COLAModel=gemma-2-9b-it, Calibration Category=Mean2026.03 | 74.31 | — | |
| ZipCalModel=gemma-2-9b-it, Compression Method=AWQ, Compression Setting (Sparsity/Bit-width)=W4A162026.03 | 74.1 | -0.12 | |
| ZipCalModel=Llama-3.1-8B-Instruct, Compression Method=GPTQ, Compression Setting (Sparsity/Bit-width)=W4A162026.03 | 74.07 | 6.46 | |
| ZipCalModel=gemma-2-9b-it, Compression Method=GPTQ, Compression Setting (Sparsity/Bit-width)=W4A162026.03 | 73.61 | -0.55 | |
| ZipCalModel=Llama-3.1-8B-Instruct, Compression Method=AWQ, Compression Setting (Sparsity/Bit-width)=W4A162026.03 | 70.5 | 1.52 | |
| COLAModel=Llama-3.1-8B-Instruct, Calibration Category=Mean2026.03 | 4.88 | — | |
| ZipCalModel=Llama-3.1-8B-Instruct, Calibration Category=Mean2026.03 | 4.63 | -0.25 | |
| ZipCalModel=Llama-3.1-8B-Instruct, Compression Method=2SSP, Compression Setting (Sparsity/Bit-width)=25%2026.03 | 4.13 | -0.75 | |
| ZipCalModel=gemma-2-9b-it, Compression Method=2SSP, Compression Setting (Sparsity/Bit-width)=25%2026.03 | 3.51 | 0.21 | |
| COLAModel=gemma-2-9b-it, Calibration Category=Mean2026.03 | 3.3 | — | |
| ZipCalModel=gemma-2-9b-it, Calibration Category=Mean2026.03 | 2.76 | -0.54 |