Zero-shot Classification on HellaSwag, PIQA, COPA, ARC-C, ARC-E, WinoGrande
80.79PIQA AccBaseline
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| BaselineModel=LLaMA3-8B, Precision=FP162025.12 | 80.79 | 79.16 | 89 | 53.41 | 77.78 | 72.85 | 75.5 | |
| SMQ DynamicModel=LLaMA2-13B2025.12 | 80.58 | 79.25 | 91 | 48.81 | 77.02 | 71.27 | 74.65 | |
| BaselineModel=LLaMA2-13B, Precision=FP162025.12 | 80.52 | 79.37 | 91 | 49.06 | 77.4 | 72.3 | 74.94 | |
| QuaRotModel=LLaMA2-13B2025.12 | 80.36 | 79.38 | 91 | 49.15 | 77.31 | 71.98 | 74.86 | |
| SMQ DynamicModel=LLaMA3-8B2025.12 | 80.09 | 78.95 | 89 | 52.39 | 77.1 | 72.45 | 75 | |
| BaselineModel=Qwen2.5-7B, Precision=FP162025.12 | 80.03 | 78.71 | 91 | 51.11 | 77.78 | 71.82 | 75.74 | |
| LLM-QATModel=LLaMA2-13B2025.12 | 80 | 77.4 | — | — | 75.3 | 71.6 | — | |
| SASQModel=LLaMA3-8B2025.12 | 79.87 | 77.57 | 88 | 51.88 | 78.07 | 71.51 | 74.48 | |
| SMQ DynamicModel=Qwen2.5-7B2025.12 | 79.65 | 78.49 | 88 | 48.89 | 75.38 | 71.27 | 73.61 | |
| SpinQuantModel=LLaMA3-8B2025.12 | 79.6 | 78.1 | — | — | 76.5 | 72.4 | — | |
| LLM-QATModel=LLaMA1-7B2025.12 | 79.4 | 75.6 | — | — | 72.3 | 69.7 | — | |
| BaselineModel=LLaMA1-7B, Precision=FP162025.12 | 79.16 | 76.2 | 84 | 44.71 | 72.93 | 69.93 | 71.16 | |
| BaselineModel=LLaMA2-7B, Precision=FP162025.12 | 79.11 | 75.99 | 87 | 46.25 | 74.58 | 69.14 | 72.18 | |
| AWQModel=LLaMA2-13B2025.12 | 79 | — | — | 47.7 | 79.04 | 73.24 | — | |
| LLM-QATModel=LLaMA3-8B2025.12 | 79 | 76 | — | — | 77.6 | 72.4 | — | |
| QuaRotModel=LLaMA2-7B2025.12 | 78.94 | 75.8 | 86 | 45.39 | 74.79 | 68.67 | 71.6 | |
| SpinQuantModel=LLaMA2-7B2025.12 | 78.9 | 74.8 | — | — | 74 | 68.9 | — | |
| SASQModel=LLaMA2-13B2025.12 | 78.89 | 77.35 | 88 | 49.66 | 76.01 | 70.32 | 73.37 | |
| SMQ DynamicModel=LLaMA1-7B2025.12 | 78.84 | 75.81 | 87 | 44.54 | 75.21 | 70.24 | 71.94 | |
| LLM-QATModel=LLaMA2-7B2025.12 | 78.2 | 74 | — | — | 73.6 | 67.7 | — | |
| SASQModel=Qwen2.5-7B2025.12 | 78.18 | 75.71 | 87 | 49.74 | 76.47 | 71.19 | 73.05 | |
| SASQModel=LLaMA1-7B2025.12 | 78.13 | 74.81 | 86 | 44.03 | 72.22 | 68.75 | 70.66 | |
| AWQModel=LLaMA1-7B2025.12 | 77.97 | — | — | 41.21 | — | 68.75 | — | |
| SASQModel=LLaMA2-7B2025.12 | 77.86 | 73.46 | 88 | 44.45 | 72.22 | 67.72 | 70.62 | |
| SMQ DynamicModel=LLaMA2-7B2025.12 | 77.8 | 76.1 | 88 | 46.15 | 74.1 | 68.58 | 71.79 | |
| AWQModel=LLaMA2-7B2025.12 | 77.31 | — | — | 43.86 | 76.14 | 68.98 | — | |
| BaselineModel=Qwen2.5-1.5B, Precision=FP162025.12 | 75.68 | 67.62 | 82 | 44.62 | 71.51 | 62.59 | 67 | |
| SMQ DynamicModel=Qwen2.5-1.5B2025.12 | 75.46 | 67.44 | 83 | 43.09 | 69.57 | 64.33 | 67.15 | |
| SASQModel=Qwen2.5-1.5B2025.12 | 74.65 | 65.36 | 77 | 44.03 | 71.55 | 65.04 | 66.27 | |
| SMQ StaticModel=LLaMA2-7B2025.12 | 60.07 | 33.55 | 85 | 27.56 | 52.15 | 64.48 | 53.8 |