Language Understanding on MMLU-K
76.37AccuracyZipCal
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ZipCalModel=Llama-3.1-8B-Instruct, Compression Method=Wanda, Compression Setting (Sparsity/Bit-width)=25%2026.03 | 76.37 | 14.35 | |
| ZipCalModel=gemma-2-9b-it, Compression Method=GPTQ, Compression Setting (Sparsity/Bit-width)=W4A162026.03 | 35.19 | 3.93 | |
| ZipCalModel=Llama-3.1-8B-Instruct, Calibration Category=Mean2026.03 | 31.26 | -4.33 | |
| ZipCalModel=gemma-2-9b-it, Calibration Category=Mean2026.03 | 21.17 | -1.6 |