Mathematical Reasoning on Minerva Math (4-shot %)
20.6Minerva Math 4-shot AccuracyLoRA FP16
Evaluation Results
| Method | Links | |
|---|---|---|
| LoRA FP16Rank=r64, BPW=16.0, Size (MB)=185.5, Time=–2026.05 | 20.6 | |
| LoRA FP16Rank=r16, BPW=16.0, Size (MB)=46.4, Time=–2026.05 | 17.56 | |
| QAT FullRank=r16, BPW=4.0, Size (MB)=11.6, Time=+7h2026.05 | 17.44 | |
| QAT FreezeRank=r16, BPW=4.0, Size (MB)=11.6, Time=+4h2026.05 | 17.1 | |
| PTQ-LoRDBARank=r16, BPW=4.0, Size (MB)=11.6, Time=+2 min2026.05 | 16.5 | |
| QAT FreezeRank=r64, BPW=1.0, Size (MB)=11.6, Time=+4h2026.05 | 16.46 | |
| QAT FullRank=r64, BPW=1.0, Size (MB)=11.6, Time=+7h2026.05 | 16.32 | |
| QAT FullRank=r16, BPW=2.0, Size (MB)=5.8, Time=+7h2026.05 | 15.34 | |
| LoRAQuant ρ=0.9Rank=r16, BPW=1.71, Size (MB)=5.0, Time=+17 min2026.05 | 15.2 | |
| QAT FreezeRank=r16, BPW=2.0, Size (MB)=5.8, Time=+4h2026.05 | 15.2 | |
| LoRAQuant ρ=0.7Rank=r16, BPW=1.42, Size (MB)=4.1, Time=+11 min2026.05 | 14.64 | |
| PTQ-LoRDBARank=r64, BPW=1.0, Size (MB)=11.6, Time=+2 min2026.05 | 14.6 | |
| QAT ScratchRank=r16, BPW=4.0, Size (MB)=11.6, Time=7h2026.05 | 14.14 | |
| LoRAQuant ρ=0.5Rank=r16, BPW=1.22, Size (MB)=3.6, Time=+8 min2026.05 | 14.1 | |
| QAT FullRank=r16, BPW=1.5, Size (MB)=4.4, Time=+7h2026.05 | 13.86 | |
| QAT FreezeRank=r16, BPW=1.5, Size (MB)=4.4, Time=+4h2026.05 | 13.38 | |
| BitDelta-rowRank=r16, BPW=2.00, Size (MB)=5.8, Time=+0 min2026.05 | 13.18 | |
| QAT ScratchRank=r64, BPW=1.0, Size (MB)=11.6, Time=7h2026.05 | 13.04 | |
| BitDelta-scalarRank=r16, BPW=1.00, Size (MB)=2.9, Time=+0 min2026.05 | 13 | |
| PTQ-LoRDBARank=r16, BPW=2.0, Size (MB)=5.8, Time=+2 min2026.05 | 12.86 | |
| QAT ScratchRank=r16, BPW=2.0, Size (MB)=5.8, Time=7h2026.05 | 12.76 | |
| BitDelta-col+distillRank=r16, BPW=1.00, Size (MB)=2.9, Time=+15 min2026.05 | 12.68 | |
| BitDelta-columnRank=r16, BPW=1.00, Size (MB)=2.9, Time=+0 min2026.05 | 12.66 | |
| QAT ScratchRank=r16, BPW=1.5, Size (MB)=4.4, Time=7h2026.05 | 12.42 | |
| PTQ-LoRDBARank=r16, BPW=1.5, Size (MB)=4.4, Time=+2 min2026.05 | 12.2 |