Zero-shot Accuracy on PIQA
81.5Zero-shot PIQA AccuracyAQLM
Evaluation results
| Method | Links | |
|---|---|---|
| AQLMModel Size=2-70, Bits=4.142024.06 | 81.5 | |
| QUIP#Model Size=2-70, Bits=42024.06 | 81.4 | |
| QUIP#Model Size=2-70, Bits=32024.06 | 81.4 | |
| QTIPModel Size=2-70, Bits=42024.06 | 81.3 | |
| FP16Model Size=2-70, Bits=162024.06 | 81.1 | |
| AQLMModel Size=2-70, Bits=3.012024.06 | 80.7 | |
| QTIPModel Size=2-70, Bits=32024.06 | 80.6 | |
| AQLMModel Size=2-70, Bits=2.072024.06 | 80.4 | |
| QTIPModel Size=2-70, Bits=22024.06 | 80.2 | |
| QUIP#Model Size=2-70, Bits=22024.06 | 79.5 | |
| QTIPModel Size=2-13, Bits=42024.06 | 78.9 | |
| QUIP#Model Size=2-13, Bits=42024.06 | 78.7 | |
| AQLMModel Size=2-13, Bits=3.032024.06 | 78.5 | |
| FP16Model Size=2-7, Bits=162024.06 | 78.5 | |
| QUIP#Model Size=2-7, Bits=42024.06 | 78.5 | |
| AQLMModel Size=2-13, Bits=3.942024.06 | 78.4 | |
| QUIP#Model Size=2-13, Bits=32024.06 | 78.4 | |
| QTIPModel Size=2-7, Bits=42024.06 | 78.4 | |
| AQLMModel Size=2-7, Bits=4.042024.06 | 78.2 | |
| QTIPModel Size=2-7, Bits=32024.06 | 78.1 | |
| QTIPModel Size=2-13, Bits=32024.06 | 78 | |
| QUIP#Model Size=2-13, Bits=22024.06 | 77.3 | |
| QTIPModel Size=2-13, Bits=22024.06 | 77.3 | |
| AQLMModel Size=2-7, Bits=3.042024.06 | 77.3 | |
| QUIP#Model Size=2-7, Bits=32024.06 | 77.3 | |
| AQLMModel Size=2-13, Bits=1.972024.06 | 75.9 | |
| QTIPModel Size=2-7, Bits=22024.06 | 75.9 | |
| QUIP#Model Size=2-7, Bits=22024.06 | 75.4 | |
| AQLMModel Size=2-7, Bits=2.022024.06 | 74.8 | |
| FP16Model Size=2-13, Bits=162024.06 | 73.5 |