Zero-shot Reasoning on PIQA, ARC, HellaSwag, and WinoGrande
82.75PIQA AccuracyFlatQuant
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| FlatQuantModel=Llama-2-70B, #Bits=W4A42026.03 | 82.75 | 80.3 | 56.14 | 83.01 | 77.9 | 76.02 | |
| SliderQuant+Model=Llama-2-70B, #Bits=W4A42026.03 | 82.75 | 81.23 | 56.57 | 83.12 | 77.93 | 76.32 | |
| W16A16 BaselineModel=Llama-2-70B, #Bits=W16A162026.03 | 82.7 | 81.02 | 57.17 | 83.81 | 77.98 | 76.54 | |
| SpinQuantModel=Llama-2-70B, #Bits=W4A42026.03 | 82.37 | 79.04 | 55.38 | 82.57 | 78.22 | 75.52 | |
| QuaRotModel=Llama-2-70B, #Bits=W4A42026.03 | 81.83 | 79.76 | 55.46 | 81.58 | 76.09 | 74.94 | |
| W16A16 BaselineModel=Llama3-8B, #Bits=W16A162026.03 | 80.79 | 77.69 | 53.41 | 79.13 | 72.77 | 72.76 | |
| W16A16 BaselineModel=Llama2-13B, #Bits=W16A162026.03 | 80.41 | 77.4 | 49.15 | 79.37 | 72.14 | 71.69 | |
| W16A16 BaselineModel=Qwen2.5-7B Instruct, #Bits=W16A162026.03 | 80.2 | 75.8 | 51.37 | 79.57 | 69.93 | 71.37 | |
| SliderQuant+Model=Llama2-13B, #Bits=W4A42026.03 | 79.96 | 77.27 | 48.95 | 77.96 | 71.98 | 71.22 | |
| SliderQuant+Model=Qwen2.5-7B Instruct, #Bits=W4A42026.03 | 79.56 | 79.05 | 52.27 | 78.66 | 69.88 | 71.88 | |
| FlatQuantModel=Llama2-13B, #Bits=W4A42026.03 | 79.38 | 76.64 | 48.04 | 77.59 | 70.24 | 70.38 | |
| SliderQuant+Model=Llama3-8B, #Bits=W4A42026.03 | 79.22 | 77.53 | 50.6 | 77.31 | 72.82 | 71.5 | |
| FlatQuantModel=Llama3-8B, #Bits=W4A42026.03 | 79.16 | 75.8 | 50 | 76.8 | 72.69 | 70.89 | |
| W16A16 BaselineModel=Llama2-7B, #Bits=W16A162026.03 | 78.84 | 74.62 | 46.42 | 75.9 | 69.46 | 69.05 | |
| SpinQuantModel=Llama2-13B, #Bits=W4A42026.03 | 78.4 | 72.43 | 43.69 | 75.52 | 68.9 | 67.79 | |
| SliderQuant+Model=Llama2-7B, #Bits=W4A42026.03 | 77.97 | 73.15 | 43.35 | 73.71 | 69.74 | 67.58 | |
| QuaRotModel=Llama2-13B, #Bits=W4A42026.03 | 77.69 | 69.95 | 42.83 | 73.54 | 67.88 | 66.38 | |
| SpinQuantModel=Llama3-8B, #Bits=W4A42026.03 | 77.37 | 74.2 | 47.27 | 74.55 | 68.51 | 68.38 | |
| FlatQuantModel=Llama2-7B, #Bits=W4A42026.03 | 77.26 | 72.05 | 43.26 | 73.64 | 69.53 | 67.15 | |
| DuQuantModel=Llama2-13B, #Bits=W4A42026.03 | 77.26 | 56.23 | 42.15 | 73.68 | 65.43 | 62.95 | |
| FlatQuantModel=Qwen2.5-7B Instruct, #Bits=W4A42026.03 | 76.93 | 77.69 | 51.71 | 78.42 | 69.53 | 70.86 | |
| QuaRotModel=Llama2-7B, #Bits=W4A42026.03 | 76.77 | 69.87 | 40.87 | 72.16 | 63.77 | 64.69 | |
| DuQuantModel=Llama3-8B, #Bits=W4A42026.03 | 76.22 | 70.41 | 43.69 | 73.87 | 67.8 | 66.4 | |
| SpinQuantModel=Llama2-7B, #Bits=W4A42026.03 | 76.17 | 69.28 | 41.72 | 72.9 | 66.06 | 65.23 | |
| DuQuantModel=Llama2-7B, #Bits=W4A42026.03 | 75.68 | 50 | 37.46 | 69.74 | 63.93 | 59.36 | |
| QuaRotModel=Llama3-8B, #Bits=W4A42026.03 | 75.14 | 68.01 | 43.34 | 72.94 | 65.82 | 65.05 | |
| AtomModel=Llama2-13B, #Bits=W4A42026.03 | 71.16 | 50.89 | 37.88 | 67.51 | 58.4 | 57.17 | |
| QLLMModel=Llama2-13B, #Bits=W4A42026.03 | 70.46 | 48.48 | 34.39 | 62.8 | 55.41 | 54.31 | |
| AtomModel=Llama2-7B, #Bits=W4A42026.03 | 69.75 | 47.35 | 34.22 | 63.21 | 56.51 | 54.21 | |
| QLLMModel=Llama2-7B, #Bits=W4A42026.03 | 67.68 | 44.4 | 30.89 | 58.45 | 56.59 | 51.6 | |
| AtomModel=Llama3-8B, #Bits=W4A42026.03 | 62.95 | 49.45 | 30.12 | 53.75 | 56.04 | 50.46 |