Text Generation Accuracy on GSM8K
86.4AccuracyLoRA + PCGrad
Evaluation Results
| Method | Links | |
|---|---|---|
| LoRA + PCGradBackbone=Qwen-14B2026.06 | 86.4 | |
| GRAINBackbone=Qwen-14B2026.06 | 85.53 | |
| LoRA + SWABackbone=Qwen-14B2026.06 | 85.45 | |
| LoRA + SAMBackbone=Qwen-14B2026.06 | 85.45 | |
| Llama 3.1 8B Instruct# Bits=BF16, Decoding Strategy=greedy2026.05 | 84.99 | |
| Llama 3.1 8B Instruct# Bits=BF16, Decoding strategy=greedy2026.05 | 84.99 | |
| LoRA + NoisyTuneBackbone=Qwen-14B2026.06 | 84.46 | |
| LoRABackbone=Qwen-14B2026.06 | 82.94 | |
| LoRA + FocalLossBackbone=Qwen-14B2026.06 | 82.18 | |
| FlexRound+LFQ# Bits=W4, PTQ Method=FlexRound, Decoding Strategy=greedy, Quantization Granularity=4-bit per-channel weight-only2026.05 | 81.8 | |
| FlexRound+LFQ# Bits=W4, Decoding strategy=greedy2026.05 | 81.8 | |
| FlexRound+LFQ+RILQ# Bits=W4, PTQ Method=FlexRound, Decoding Strategy=greedy, Quantization Granularity=4-bit per-channel weight-only2026.05 | 81.43 | |
| FlexRound# Bits=W4, PTQ Method=FlexRound, Decoding Strategy=greedy, Quantization Granularity=4-bit per-channel weight-only2026.05 | 81.35 | |
| FlexRound# Bits=W4, Decoding strategy=greedy2026.05 | 81.35 | |
| FlexRound+RILQ# Bits=W4, PTQ Method=FlexRound, Decoding Strategy=greedy, Quantization Granularity=4-bit per-channel weight-only2026.05 | 80.52 | |
| FlexRound+RILQ# Bits=W4, Decoding strategy=greedy2026.05 | 80.52 | |
| GRAINBackbone=Ministral-14B2026.06 | 79.85 | |
| OmniQuant+LFQ# Bits=W4, PTQ Method=OmniQuant, Decoding Strategy=greedy, Quantization Granularity=4-bit per-channel weight-only2026.05 | 79.76 | |
| OmniQuant+LFQ# Bits=W4, Decoding strategy=greedy2026.05 | 79.76 | |
| LoRA + SAMBackbone=Ministral-14B2026.06 | 79.63 | |
| OmniQuant+LFQ+RILQ# Bits=W4, PTQ Method=OmniQuant, Decoding Strategy=greedy, Quantization Granularity=4-bit per-channel weight-only2026.05 | 79.45 | |
| LoRABackbone=Ministral-14B2026.06 | 79.24 | |
| LoRA + PCGradBackbone=Ministral-14B2026.06 | 79.12 | |
| OmniQuant+RILQ# Bits=W4, PTQ Method=OmniQuant, Decoding Strategy=greedy, Quantization Granularity=4-bit per-channel weight-only2026.05 | 78.85 | |
| OmniQuant+RILQ# Bits=W4, Decoding strategy=greedy2026.05 | 78.85 | |
| LoRA + NoisyTuneBackbone=Ministral-14B2026.06 | 78.77 | |
| LoRA + SWABackbone=Ministral-14B2026.06 | 78.77 | |
| OmniQuant# Bits=W4, PTQ Method=OmniQuant, Decoding Strategy=greedy, Quantization Granularity=4-bit per-channel weight-only2026.05 | 78.17 | |
| OmniQuant# Bits=W4, Decoding strategy=greedy2026.05 | 78.17 | |
| LoRA + FocalLossBackbone=Ministral-14B2026.06 | 76.42 | |
| GRAINBackbone=Qwen-7B2026.06 | 74.77 | |
| LoRA + PCGradBackbone=Qwen-7B2026.06 | 74.63 | |
| Block-AP+LFQ# Bits=W4, PTQ Method=Block-AP, Decoding Strategy=greedy, Quantization Granularity=4-bit per-channel weight-only2026.05 | 74.45 | |
| Block-AP+LFQ# Bits=W4, Decoding strategy=greedy2026.05 | 74.45 | |
| LoRA + NoisyTuneBackbone=Qwen-7B2026.06 | 74.43 | |
| Block-AP+LFQ+RILQ# Bits=W4, PTQ Method=Block-AP, Decoding Strategy=greedy, Quantization Granularity=4-bit per-channel weight-only2026.05 | 74.22 | |
| LoRA + FocalLossBackbone=Qwen-7B2026.06 | 73.98 | |
| Block-AP+RILQ# Bits=W4, PTQ Method=Block-AP, Decoding Strategy=greedy, Quantization Granularity=4-bit per-channel weight-only2026.05 | 73.92 | |
| Block-AP+RILQ# Bits=W4, Decoding strategy=greedy2026.05 | 73.92 | |
| LoRA + SWABackbone=Qwen-7B2026.06 | 73.9 | |
| Block-AP# Bits=W4, PTQ Method=Block-AP, Decoding Strategy=greedy, Quantization Granularity=4-bit per-channel weight-only2026.05 | 73.84 | |
| Block-AP# Bits=W4, Decoding strategy=greedy2026.05 | 73.84 | |
| LoRA + SAMBackbone=Qwen-7B2026.06 | 72.88 | |
| LoRABackbone=Qwen-7B2026.06 | 72.87 | |
| DeLaskModel=LLaMA3-8B, Decoding=DeLask2026.05 | 55.42 | |
| UADModel=LLaMA3-8B, Decoding=UAD2026.05 | 55.02 | |
| RegularModel=LLaMA3-8B, Decoding=Regular2026.05 | 54.89 | |
| DOLAModel=LLaMA3-8B, Decoding=DOLA2026.05 | 54.74 | |
| ITIModel=LLaMA3-8B, Decoding=ITI2026.05 | 54.36 | |
| ADModel=LLaMA3-8B, Decoding=AD2026.05 | 53.39 | |
| GRAINBackbone=Mistral-7B2026.06 | 49.67 | |
| LoRA + SWABackbone=Mistral-7B2026.06 | 48.65 | |
| LoRA + NoisyTuneBackbone=Mistral-7B2026.06 | 48.2 | |
| LoRA + PCGradBackbone=Mistral-7B2026.06 | 48.14 | |
| LoRABackbone=Mistral-7B2026.06 | 47.95 | |
| LoRA + FocalLossBackbone=Mistral-7B2026.06 | 47.53 | |
| LoRA + SAMBackbone=Mistral-7B2026.06 | 46.49 | |
| DeLaskModel=LLaMA2-7B, Decoding=DeLask2026.05 | 15.04 | |
| UADModel=LLaMA2-7B, Decoding=UAD2026.05 | 14.86 | |
| ITIModel=LLaMA2-7B, Decoding=ITI2026.05 | 14.74 | |
| RegularModel=LLaMA2-7B, Decoding=Regular2026.05 | 14.7 | |
| ADModel=LLaMA2-7B, Decoding=AD2026.05 | 13.76 | |
| DOLAModel=LLaMA2-7B, Decoding=DOLA2026.05 | 13.34 |