Mathematics on Minerva Math (4-shot)
15.524-shot Performance (%)LoRA FP16
Evaluation Results
| Method | Links | |
|---|---|---|
| LoRA FP16Rank=r64, BPW=16.0, Size (MB)=305.0, Time=–2026.05 | 15.52 | |
| LoRA FP16Rank=r16, BPW=16.0, Size (MB)=76.3, Time=–2026.05 | 13.17 | |
| QAT FullRank=r16, BPW=4.0, Size (MB)=19.1, Time=+11h2026.05 | 12.28 | |
| QAT FullRank=r64, BPW=1.0, Size (MB)=19.1, Time=+15h2026.05 | 12.04 | |
| QAT FreezeRank=r64, BPW=1.0, Size (MB)=19.1, Time=+15h2026.05 | 11.38 | |
| PTQ-LoRDBARank=r16, BPW=4.0, Size (MB)=19.1, Time=+2 min2026.05 | 10.04 | |
| QAT FullRank=r16, BPW=1.0, Size (MB)=7.2, Time=+11h2026.05 | 9.68 | |
| QAT FullRank=r16, BPW=2.0, Size (MB)=9.5, Time=+11h2026.05 | 9.68 | |
| LoRAQuant ρ=0.9Rank=r16, BPW=1.68, Size (MB)=8.0, Time=+55 min2026.05 | 8.94 | |
| QAT ScratchRank=r16, BPW=4.0, Size (MB)=19.1, Time=9h2026.05 | 8.5 | |
| QAT FreezeRank=r16, BPW=1.0, Size (MB)=7.2, Time=+11h2026.05 | 8.18 | |
| PTQ-LoRDBARank=r64, BPW=1.0, Size (MB)=19.1, Time=+2 min2026.05 | 7.88 | |
| QAT ScratchRank=r16, BPW=2.0, Size (MB)=9.5, Time=8h2026.05 | 7.32 | |
| LoRAQuant ρ=0.5Rank=r16, BPW=1.20, Size (MB)=5.7, Time=+36 min2026.05 | 7.18 | |
| BitDelta-rowRank=r16, BPW=2.00, Size (MB)=9.5, Time=+0 min2026.05 | 6.72 | |
| BitDelta-col+distillRank=r16, BPW=1.00, Size (MB)=4.8, Time=+17 min2026.05 | 6.46 | |
| BitDelta-scalarRank=r16, BPW=1.00, Size (MB)=4.8, Time=+0 min2026.05 | 6.38 | |
| PTQ-LoRDBARank=r16, BPW=2.0, Size (MB)=9.5, Time=+2 min2026.05 | 6.36 | |
| BitDelta-columnRank=r16, BPW=1.00, Size (MB)=4.8, Time=+0 min2026.05 | 6.28 | |
| PTQ-LoRDBARank=r16, BPW=1.0, Size (MB)=7.2, Time=+2 min2026.05 | 5.74 | |
| QAT ScratchRank=r64, BPW=1.0, Size (MB)=19.1, Time=8h2026.05 | 5.74 |