Zero-shot Classification on Evaluation Suite (ARCC, ARCE, BOOLQ, HSWAG, PIQA)
70.83Average AccuracyGOOGLE QAT
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| GOOGLE QATQuantization Type=QAT, Bits=4.52025.05 | 70.83 | 54.52 | 79.76 | 86.82 | 54.77 | 78.29 | |
| BF16Quantization Type=NONE, Bits=162025.05 | 70.22 | 54.01 | 78.79 | 87.25 | 54.27 | 76.77 | |
| YAQA-AQuantization Type=PTQ, Bits=42025.05 | 70.12 | 53.9 | 78.83 | 87.09 | 53.68 | 77.09 | |
| YAQA-BQuantization Type=PTQ, Bits=42025.05 | 69.9 | 54.1 | 78.66 | 86.91 | 54.13 | 75.68 |