Zero-shot Evaluation on PIQA, HellaSwag, COPA, RTE, OpenBookQA, and LAMBADA
75.97Average ScoreBF16
Evaluation results
| Method | Links | |
|---|---|---|
| BF16config=BF16 (ref), bpw=16.000, model=Mistral-7B-Instruct-v0.32026.05 | 75.97 | |
| SmoothQuantconfig=W8A8, bpw=8.128, model=Mistral-7B-Instruct-v0.32026.05 | 75.97 | |
| QAM-W-5.5config=QAM-W-5.5, bpw=5.506, model=Mistral-7B-Instruct-v0.32026.05 | 75.51 | |
| AWQconfig=W4A16 g128, bpw=4.128, model=Mistral-7B-Instruct-v0.32026.05 | 74.04 | |
| GPTQ W4A16 g128bits-per-weight=4.122026.05 | 73.63 | |
| BF16 (ref)bits-per-weight=16.002026.05 | 73.04 | |
| SmoothQuant W8A8bits-per-weight=8.132026.05 | 73.04 | |
| GPTQconfig=W4A16 g128, bpw=4.125, model=Mistral-7B-Instruct-v0.32026.05 | 72.9 | |
| QAM-W-5.5bits-per-weight=5.502026.05 | 72.75 | |
| QTIP 4-bit (relaxml)bpw=4.002026.05 | 70.27 | |
| SmoothQuant W8A8bpw=8.132026.05 | 70.09 | |
| QAM-W-5.5bpw=5.512026.05 | 70.08 | |
| QAM-W-4bpw=4.002026.05 | 69.93 | |
| BF16 (ref)bpw=16.002026.05 | 69.9 | |
| AQLM 2-bit 1×16 (ISTA-DASLab)bpw=2.032026.05 | 67.82 | |
| QTIP 2-bit (relaxml)bpw=2.002026.05 | 67.64 |