Mathematical Reasoning on GSM8K (8-shot exact match)
61.56GSM8K 8-shot AccuracyLoRA FP16
Evaluation Results
| Method | Links | |
|---|---|---|
| LoRA FP16Rank=r64, BPW=16.0, Size (MB)=185.5, Time=–2026.05 | 61.56 | |
| LoRA FP16Rank=r16, BPW=16.0, Size (MB)=46.4, Time=–2026.05 | 59.36 | |
| QAT FreezeRank=r16, BPW=4.0, Size (MB)=11.6, Time=+4h2026.05 | 58.98 | |
| QAT FreezeRank=r64, BPW=1.0, Size (MB)=11.6, Time=+4h2026.05 | 57.16 | |
| QAT FreezeRank=r16, BPW=2.0, Size (MB)=5.8, Time=+4h2026.05 | 57.01 | |
| QAT FullRank=r64, BPW=1.0, Size (MB)=11.6, Time=+7h2026.05 | 56.71 | |
| QAT FullRank=r16, BPW=4.0, Size (MB)=11.6, Time=+7h2026.05 | 56.63 | |
| QAT FullRank=r16, BPW=2.0, Size (MB)=5.8, Time=+7h2026.05 | 55.42 | |
| PTQ-LoRDBARank=r16, BPW=4.0, Size (MB)=11.6, Time=+2 min2026.05 | 55.04 | |
| QAT ScratchRank=r16, BPW=4.0, Size (MB)=11.6, Time=7h2026.05 | 54.66 | |
| QAT FullRank=r16, BPW=1.5, Size (MB)=4.4, Time=+7h2026.05 | 52.77 | |
| LoRAQuant ρ=0.9Rank=r16, BPW=1.71, Size (MB)=5.0, Time=+17 min2026.05 | 51.63 | |
| QAT FreezeRank=r16, BPW=1.5, Size (MB)=4.4, Time=+4h2026.05 | 50.27 | |
| QAT ScratchRank=r16, BPW=2.0, Size (MB)=5.8, Time=7h2026.05 | 50.19 | |
| PTQ-LoRDBARank=r64, BPW=1.0, Size (MB)=11.6, Time=+2 min2026.05 | 49.81 | |
| QAT ScratchRank=r64, BPW=1.0, Size (MB)=11.6, Time=7h2026.05 | 48.29 | |
| PTQ-LoRDBARank=r16, BPW=2.0, Size (MB)=5.8, Time=+2 min2026.05 | 48.22 | |
| LoRAQuant ρ=0.7Rank=r16, BPW=1.42, Size (MB)=4.1, Time=+11 min2026.05 | 47.31 | |
| QAT ScratchRank=r16, BPW=1.5, Size (MB)=4.4, Time=7h2026.05 | 45.56 | |
| LoRAQuant ρ=0.5Rank=r16, BPW=1.22, Size (MB)=3.6, Time=+8 min2026.05 | 45.26 | |
| BitDelta-rowRank=r16, BPW=2.00, Size (MB)=5.8, Time=+0 min2026.05 | 43.29 | |
| BitDelta-columnRank=r16, BPW=1.00, Size (MB)=2.9, Time=+0 min2026.05 | 42.68 | |
| BitDelta-col+distillRank=r16, BPW=1.00, Size (MB)=2.9, Time=+15 min2026.05 | 42.61 | |
| BitDelta-scalarRank=r16, BPW=1.00, Size (MB)=2.9, Time=+0 min2026.05 | 42.46 | |
| PTQ-LoRDBARank=r16, BPW=1.5, Size (MB)=4.4, Time=+2 min2026.05 | 41.32 |