Zero-shot Accuracy on WinoGrande
77.3Zero-shot AccuracyQUIP#
Evaluation Results
| Method | Links | |
|---|---|---|
| QUIP#Model Size=2-70, Bits=42024.06 | 77.3 | |
| FP16Model Size=2-70, Bits=162024.06 | 77 | |
| QTIPModel Size=2-70, Bits=32024.06 | 77 | |
| QTIPModel Size=2-70, Bits=42024.06 | 76.9 | |
| AQLMModel Size=2-70, Bits=4.142024.06 | 76.5 | |
| QUIP#Model Size=2-70, Bits=32024.06 | 76.1 | |
| AQLMModel Size=2-70, Bits=2.072024.06 | 75.9 | |
| AQLMModel Size=2-70, Bits=3.012024.06 | 75.3 | |
| QTIPModel Size=2-70, Bits=22024.06 | 75.1 | |
| QUIP#Model Size=2-70, Bits=22024.06 | 74.6 | |
| AQLMModel Size=2-13, Bits=3.942024.06 | 69.9 | |
| QTIPModel Size=2-13, Bits=42024.06 | 69.9 | |
| FP16Model Size=2-13, Bits=162024.06 | 69.6 | |
| QUIP#Model Size=2-13, Bits=42024.06 | 69.6 | |
| QTIPModel Size=2-13, Bits=32024.06 | 69.5 | |
| QUIP#Model Size=2-13, Bits=32024.06 | 69.1 | |
| AQLMModel Size=2-13, Bits=3.032024.06 | 68.8 | |
| AQLMModel Size=2-13, Bits=1.972024.06 | 68.8 | |
| QTIPModel Size=2-13, Bits=22024.06 | 67.6 | |
| QUIP#Model Size=2-13, Bits=22024.06 | 67.4 | |
| QUIP#Model Size=2-7, Bits=42024.06 | 67.4 | |
| FP16Model Size=2-7, Bits=162024.06 | 67.3 | |
| AQLMModel Size=2-7, Bits=4.042024.06 | 67.3 | |
| QTIPModel Size=2-7, Bits=42024.06 | 67.1 | |
| QTIPModel Size=2-7, Bits=32024.06 | 66.9 | |
| QUIP#Model Size=2-7, Bits=32024.06 | 66.5 | |
| AQLMModel Size=2-7, Bits=2.022024.06 | 65.7 | |
| AQLMModel Size=2-7, Bits=3.042024.06 | 65.4 | |
| QUIP#Model Size=2-7, Bits=22024.06 | 64.9 | |
| QTIPModel Size=2-7, Bits=22024.06 | 64.7 |