List your company
ResearchFormerly WizWand

Commonsense Reasoning on HSwag

53.27Normalized PLL ScoreBF16 Baseline
20.853229.269137.68546.1009Jul 24, 2025Aug 17, 2025Sep 10, 2025Oct 4, 2025Oct 28, 2025Nov 21, 2025Dec 16, 2025
Updated 8mo ago

Evaluation results

MethodLinks
BF16 BaselineAvg. Bitwidth=16.000, Zero-shot=true2025.0753.27
SSQR-3%Avg. Bitwidth=5.085, Zero-shot=true2025.0753.03
HPTQAvg. Bitwidth=4.125, Zero-shot=true2025.0753.02
SSQR-5%Avg. Bitwidth=5.725, Zero-shot=true2025.0752.88
SSQR-4%Avg. Bitwidth=5.405, Zero-shot=true2025.0752.8
SSQR-2%Avg. Bitwidth=4.765, Zero-shot=true2025.0752.49
SSQR-1%Avg. Bitwidth=4.445, Zero-shot=true2025.0752.37
GPTQAvg. Bitwidth=4.125, Zero-shot=true2025.0752.29
HPTQAvg. Bitwidth=3.125, Zero-shot=true2025.0751.76
SSQR-5%Avg. Bitwidth=4.725, Zero-shot=true2025.0751.32
SSQR-4%Avg. Bitwidth=4.405, Zero-shot=true2025.0751.08
SSQR-3%Avg. Bitwidth=4.085, Zero-shot=true2025.0750.98
SSQR-2%Avg. Bitwidth=3.765, Zero-shot=true2025.0750.88
SSQR-1%Avg. Bitwidth=3.445, Zero-shot=true2025.0750.55
GPTQAvg. Bitwidth=3.125, Zero-shot=true2025.0747.25
HPTQAvg. Bitwidth=2.125, Zero-shot=true2025.0744.54
SSQR-5%Avg. Bitwidth=3.725, Zero-shot=true2025.0741.37
SSQR-4%Avg. Bitwidth=3.405, Zero-shot=true2025.0740.66
SSQR-3%Avg. Bitwidth=3.085, Zero-shot=true2025.0739.87
SSQR-2%Avg. Bitwidth=2.765, Zero-shot=true2025.0734.12
SSQR-1%Avg. Bitwidth=2.445, Zero-shot=true2025.0731.14
GPTQAvg. Bitwidth=2.125, Zero-shot=true2025.0728.06
DualRepetitions=32, Alpha (α)=3/42025.1227.8
DualRepetitions=128, Alpha (α)=1/82025.1225
Masked-DiffusionRepetitions=32, Alpha (α)=02025.1224.1
Masked-DiffusionRepetitions=128, Alpha (α)=0:12025.1222.1