Zero-shot Reasoning on Reasoning Suite (PIQA, ARC, HS, WG, BoolQ, MMLU)
80.2PIQA AccuracySliderQuant
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| SliderQuant#Bits=W3A162026.03 | 80.2 | 79 | 54.61 | 74.8 | 70.48 | 87.61 | 74.63 | 74.48 | |
| Baseline#Bits=W16A162026.03 | 80.14 | 79.25 | 56.23 | 77.66 | 69.93 | 88.56 | 77.74 | 75.64 | |
| SliderQuant#Bits=W4A162026.03 | 79.87 | 77.44 | 55.2 | 76.75 | 70.96 | 87.92 | 76.83 | 75 | |
| OmniQuant#Bits=W4A162026.03 | 79.43 | 77.23 | 52.22 | 76.36 | 69.46 | 87.68 | 76.21 | 74.08 | |
| OmniQuant#Bits=W3A162026.03 | 79 | 76.73 | 52.73 | 74.77 | 67.25 | 86.94 | 73.63 | 73.01 | |
| SliderQuant#Bits=W4A42026.03 | 75.52 | 70.79 | 46.67 | 68.82 | 63.14 | 82.26 | 60.8 | 66.86 | |
| SliderQuant#Bits=W2A162026.03 | 72.36 | 67.51 | 40.53 | 60.79 | 62.04 | 77.61 | 47.19 | 61.15 | |
| OmniQuant#Bits=W2A162026.03 | 70.78 | 54.97 | 34.04 | 56.59 | 54.46 | 69.63 | 32.67 | 53.31 | |
| OmniQuant#Bits=W4A42026.03 | 67.41 | 55.64 | 33.28 | 52.34 | 52.09 | 64.86 | 25.17 | 50.11 |