Commonsense Reasoning on HSwag
53.27Normalized PLL ScoreBF16 Baseline
Evaluation Results
| Method | Links | |
|---|---|---|
| BF16 BaselineAvg. Bitwidth=16.000, Zero-shot=true2025.07 | 53.27 | |
| SSQR-3%Avg. Bitwidth=5.085, Zero-shot=true2025.07 | 53.03 | |
| HPTQAvg. Bitwidth=4.125, Zero-shot=true2025.07 | 53.02 | |
| SSQR-5%Avg. Bitwidth=5.725, Zero-shot=true2025.07 | 52.88 | |
| SSQR-4%Avg. Bitwidth=5.405, Zero-shot=true2025.07 | 52.8 | |
| SSQR-2%Avg. Bitwidth=4.765, Zero-shot=true2025.07 | 52.49 | |
| SSQR-1%Avg. Bitwidth=4.445, Zero-shot=true2025.07 | 52.37 | |
| GPTQAvg. Bitwidth=4.125, Zero-shot=true2025.07 | 52.29 | |
| HPTQAvg. Bitwidth=3.125, Zero-shot=true2025.07 | 51.76 | |
| SSQR-5%Avg. Bitwidth=4.725, Zero-shot=true2025.07 | 51.32 | |
| SSQR-4%Avg. Bitwidth=4.405, Zero-shot=true2025.07 | 51.08 | |
| SSQR-3%Avg. Bitwidth=4.085, Zero-shot=true2025.07 | 50.98 | |
| SSQR-2%Avg. Bitwidth=3.765, Zero-shot=true2025.07 | 50.88 | |
| SSQR-1%Avg. Bitwidth=3.445, Zero-shot=true2025.07 | 50.55 | |
| GPTQAvg. Bitwidth=3.125, Zero-shot=true2025.07 | 47.25 | |
| HPTQAvg. Bitwidth=2.125, Zero-shot=true2025.07 | 44.54 | |
| SSQR-5%Avg. Bitwidth=3.725, Zero-shot=true2025.07 | 41.37 | |
| SSQR-4%Avg. Bitwidth=3.405, Zero-shot=true2025.07 | 40.66 | |
| SSQR-3%Avg. Bitwidth=3.085, Zero-shot=true2025.07 | 39.87 | |
| SSQR-2%Avg. Bitwidth=2.765, Zero-shot=true2025.07 | 34.12 | |
| SSQR-1%Avg. Bitwidth=2.445, Zero-shot=true2025.07 | 31.14 | |
| GPTQAvg. Bitwidth=2.125, Zero-shot=true2025.07 | 28.06 | |
| DualRepetitions=32, Alpha (α)=3/42025.12 | 27.8 | |
| DualRepetitions=128, Alpha (α)=1/82025.12 | 25 | |
| Masked-DiffusionRepetitions=32, Alpha (α)=02025.12 | 24.1 | |
| Masked-DiffusionRepetitions=128, Alpha (α)=0:12025.12 | 22.1 |