List your company
ResearchFormerly WizWand

Mathematical Reasoning on GSM8K (8-shot exact match)

61.56GSM8K 8-shot AccuracyLoRA FP16
40.510445.975251.4456.9048May 22, 2026
Updated 5mo ago

Evaluation results

MethodLinks
LoRA FP16Rank=r64, BPW=16.0, Size (MB)=185.5, Time=–2026.0561.56
LoRA FP16Rank=r16, BPW=16.0, Size (MB)=46.4, Time=–2026.0559.36
QAT FreezeRank=r16, BPW=4.0, Size (MB)=11.6, Time=+4h2026.0558.98
QAT FreezeRank=r64, BPW=1.0, Size (MB)=11.6, Time=+4h2026.0557.16
QAT FreezeRank=r16, BPW=2.0, Size (MB)=5.8, Time=+4h2026.0557.01
QAT FullRank=r64, BPW=1.0, Size (MB)=11.6, Time=+7h2026.0556.71
QAT FullRank=r16, BPW=4.0, Size (MB)=11.6, Time=+7h2026.0556.63
QAT FullRank=r16, BPW=2.0, Size (MB)=5.8, Time=+7h2026.0555.42
PTQ-LoRDBARank=r16, BPW=4.0, Size (MB)=11.6, Time=+2 min2026.0555.04
QAT ScratchRank=r16, BPW=4.0, Size (MB)=11.6, Time=7h2026.0554.66
QAT FullRank=r16, BPW=1.5, Size (MB)=4.4, Time=+7h2026.0552.77
LoRAQuant ρ=0.9Rank=r16, BPW=1.71, Size (MB)=5.0, Time=+17 min2026.0551.63
QAT FreezeRank=r16, BPW=1.5, Size (MB)=4.4, Time=+4h2026.0550.27
QAT ScratchRank=r16, BPW=2.0, Size (MB)=5.8, Time=7h2026.0550.19
PTQ-LoRDBARank=r64, BPW=1.0, Size (MB)=11.6, Time=+2 min2026.0549.81
QAT ScratchRank=r64, BPW=1.0, Size (MB)=11.6, Time=7h2026.0548.29
PTQ-LoRDBARank=r16, BPW=2.0, Size (MB)=5.8, Time=+2 min2026.0548.22
LoRAQuant ρ=0.7Rank=r16, BPW=1.42, Size (MB)=4.1, Time=+11 min2026.0547.31
QAT ScratchRank=r16, BPW=1.5, Size (MB)=4.4, Time=7h2026.0545.56
LoRAQuant ρ=0.5Rank=r16, BPW=1.22, Size (MB)=3.6, Time=+8 min2026.0545.26
BitDelta-rowRank=r16, BPW=2.00, Size (MB)=5.8, Time=+0 min2026.0543.29
BitDelta-columnRank=r16, BPW=1.00, Size (MB)=2.9, Time=+0 min2026.0542.68
BitDelta-col+distillRank=r16, BPW=1.00, Size (MB)=2.9, Time=+15 min2026.0542.61
BitDelta-scalarRank=r16, BPW=1.00, Size (MB)=2.9, Time=+0 min2026.0542.46
PTQ-LoRDBARank=r16, BPW=1.5, Size (MB)=4.4, Time=+2 min2026.0541.32