Question Answering on OpenBookQA (Accuracy and Delta)
47.9AccuracyZipCal
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ZipCalModel=Llama-3.1-8B-Instruct, Compression Method=Wanda, Compression Setting (Sparsity/Bit-width)=25%2026.03 | 47.9 | 1.11 | |
| ZipCalModel=gemma-2-9b-it, Compression Method=AWQ, Compression Setting (Sparsity/Bit-width)=W4A162026.03 | 46.1 | 0.93 | |
| ZipCalModel=Llama-3.1-8B-Instruct, Calibration Category=Mean2026.03 | 0.4072 | 0.47 | |
| ZipCalModel=gemma-2-9b-it, Calibration Category=Mean2026.03 | 0.3846 | 0.32 |