Language Modeling on Wiki, C4, and Pile
7.15Average PerplexityDense
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DenseModel=Llama-3.1-8B2026.03 | 7.15 | — | |
| DenseBackbone=Llama-3.1-8B2026.03 | 7.15 | — | |
| AWQ (W4A16)Model=Llama-3.1-8B, Calibration Method=COLA2026.03 | 7.35 | — | |
| COLABackbone=Llama-3.1-8B, Compression=AWQ (W4A16)2026.03 | 7.35 | — | |
| AWQ (W4A16)Model=Llama-3.1-8B, Calibration Method=ZipCal2026.03 | 7.4 | — | |
| AWQ (W4A16)Model=Llama-3.1-8B, Calibration Method=ZipCal (Multi-Domain)2026.03 | 7.4 | 0.05 | |
| ZipCal-SDBackbone=Llama-3.1-8B, Compression=AWQ (W4A16)2026.03 | 7.4 | — | |
| ZipCal-MDBackbone=Llama-3.1-8B, Compression=AWQ (W4A16)2026.03 | 7.4 | 0.05 | |
| Wanda (25%)Model=Llama-3.1-8B, Calibration Method=ZipCal (Multi-Domain)2026.03 | 7.67 | -0.1 | |
| ZipCal-MDBackbone=Llama-3.1-8B, Compression=Wanda (25%)2026.03 | 7.67 | -0.1 | |
| Wanda (25%)Model=Llama-3.1-8B, Calibration Method=ZipCal2026.03 | 7.76 | — | |
| ZipCal-SDBackbone=Llama-3.1-8B, Compression=Wanda (25%)2026.03 | 7.76 | — | |
| Wanda (25%)Model=Llama-3.1-8B, Calibration Method=COLA2026.03 | 7.77 | — | |
| COLABackbone=Llama-3.1-8B, Compression=Wanda (25%)2026.03 | 7.77 | — | |
| DenseModel=Gemma-2-9B2026.03 | 7.93 | — | |
| DenseBackbone=gemma-2-9B2026.03 | 7.93 | — | |
| GPTQ (W4A16)Model=Llama-3.1-8B, Calibration Method=COLA2026.03 | 8.09 | — | |
| GPTQ (W4A16)Model=Llama-3.1-8B, Calibration Method=ZipCal2026.03 | 8.09 | — | |
| COLABackbone=Llama-3.1-8B, Compression=GPTQ (W4A16)2026.03 | 8.09 | — | |
| ZipCal-SDBackbone=Llama-3.1-8B, Compression=GPTQ (W4A16)2026.03 | 8.09 | — | |
| Wanda (25%)Model=Gemma-2-9B, Calibration Method=COLA2026.03 | 8.13 | — | |
| Wanda (25%)Model=Gemma-2-9B, Calibration Method=ZipCal (Multi-Domain)2026.03 | 8.13 | 0 | |
| COLABackbone=gemma-2-9B, Compression=Wanda (25%)2026.03 | 8.13 | — | |
| ZipCal-MDBackbone=gemma-2-9B, Compression=Wanda (25%)2026.03 | 8.13 | 0 | |
| Wanda (25%)Model=Gemma-2-9B, Calibration Method=ZipCal2026.03 | 8.22 | — | |
| ZipCal-SDBackbone=gemma-2-9B, Compression=Wanda (25%)2026.03 | 8.22 | — | |
| GPTQ (W4A16)Model=Gemma-2-9B, Calibration Method=COLA2026.03 | 8.27 | — | |
| GPTQ (W4A16)Model=Gemma-2-9B, Calibration Method=ZipCal2026.03 | 8.27 | — | |
| COLABackbone=gemma-2-9B, Compression=GPTQ (W4A16)2026.03 | 8.27 | — | |
| ZipCal-SDBackbone=gemma-2-9B, Compression=GPTQ (W4A16)2026.03 | 8.27 | — | |
| GPTQ (W4A16)Model=Llama-3.1-8B, Calibration Method=ZipCal (Multi-Domain)2026.03 | 8.67 | 0.58 | |
| GPTQ (W4A16)Model=Gemma-2-9B, Calibration Method=ZipCal (Multi-Domain)2026.03 | 8.67 | 0.4 | |
| ZipCal-MDBackbone=Llama-3.1-8B, Compression=GPTQ (W4A16)2026.03 | 8.67 | 0 | |
| ZipCal-MDBackbone=gemma-2-9B, Compression=GPTQ (W4A16)2026.03 | 8.67 | 0 | |
| 2SSP (25%)Model=Gemma-2-9B, Calibration Method=ZipCal2026.03 | 12.41 | — | |
| ZipCal-SDBackbone=gemma-2-9B, Compression=2SSP (25%)2026.03 | 12.41 | -0.33 | |
| 2SSP (25%)Model=Gemma-2-9B, Calibration Method=COLA2026.03 | 12.74 | — | |
| COLABackbone=gemma-2-9B, Compression=2SSP (25%)2026.03 | 12.74 | — | |
| 2SSP (25%)Model=Gemma-2-9B, Calibration Method=ZipCal (Multi-Domain)2026.03 | 14.18 | 1.44 | |
| ZipCal-MDBackbone=gemma-2-9B, Compression=2SSP (25%)2026.03 | 14.18 | — | |
| AWQ (W4A16)Model=Gemma-2-9B, Calibration Method=ZipCal (Multi-Domain)2026.03 | 15.73 | -0.11 | |
| ZipCal-MDBackbone=gemma-2-9B, Compression=AWQ (W4A16)2026.03 | 15.73 | -0.11 | |
| AWQ (W4A16)Model=Gemma-2-9B, Calibration Method=ZipCal2026.03 | 15.75 | — | |
| ZipCal-SDBackbone=gemma-2-9B, Compression=AWQ (W4A16)2026.03 | 15.75 | — | |
| 2SSP (25%)Model=Llama-3.1-8B, Calibration Method=ZipCal2026.03 | 15.81 | — | |
| ZipCal-SDBackbone=Llama-3.1-8B, Compression=2SSP (25%)2026.03 | 15.81 | -0.51 | |
| AWQ (W4A16)Model=Gemma-2-9B, Calibration Method=COLA2026.03 | 15.84 | — | |
| COLABackbone=gemma-2-9B, Compression=AWQ (W4A16)2026.03 | 15.84 | — | |
| 2SSP (25%)Model=Llama-3.1-8B, Calibration Method=COLA2026.03 | 16.32 | — | |
| COLABackbone=Llama-3.1-8B, Compression=2SSP (25%)2026.03 | 16.32 | — | |
| 2SSP (25%)Model=Llama-3.1-8B, Calibration Method=ZipCal (Multi-Domain)2026.03 | 18.52 | 2.2 | |
| ZipCal-MDBackbone=Llama-3.1-8B, Compression=2SSP (25%)2026.03 | 18.52 | — |