Summarization on XSum (ROUGE-L)
27.45ROUGE-LZO-Adam
Evaluation Results
| Method | Links | |
|---|---|---|
| ZO-AdamBackbone=Qwen-3, Model Size=8B, Fine-tuning=16-rank LoRA2026.05 | 27.45 | |
| RAdaZOBackbone=Qwen-3, Model Size=8B, Fine-tuning=16-rank LoRA2026.05 | 27.34 | |
| MEAZOBackbone=Qwen-3, Model Size=8B, Fine-tuning=16-rank LoRA2026.05 | 27.31 | |
| ZO-SGDBackbone=Qwen-3, Model Size=8B, Fine-tuning=16-rank LoRA2026.05 | 26.96 | |
| RAdaZOBackbone=Llama-3.2, Model Size=1B, Fine-tuning=16-rank LoRA2026.05 | 22.39 | |
| MEAZOBackbone=Llama-3.2, Model Size=1B, Fine-tuning=16-rank LoRA2026.05 | 22.39 | |
| ZO-AdamBackbone=Llama-3.2, Model Size=1B, Fine-tuning=16-rank LoRA2026.05 | 22.36 | |
| ZO-SGDBackbone=Llama-3.2, Model Size=1B, Fine-tuning=16-rank LoRA2026.05 | 21.99 | |
| RAdaZOBackbone=Qwen-3, Model Size=0.6B, Fine-tuning=16-rank LoRA2026.05 | 20.12 | |
| ZO-SGDBackbone=Qwen-3, Model Size=0.6B, Fine-tuning=16-rank LoRA2026.05 | 20.08 | |
| ZO-AdamBackbone=Qwen-3, Model Size=0.6B, Fine-tuning=16-rank LoRA2026.05 | 20.08 | |
| MEAZOBackbone=Qwen-3, Model Size=0.6B, Fine-tuning=16-rank LoRA2026.05 | 20.08 | |
| BaseBackbone=Qwen-3, Model Size=8B, Fine-tuning=None (Before tuning)2026.05 | 18.6 | |
| ZO-SGDBackbone=Llama-3.2 1B, Tuning=20-token prompt tuning2026.05 | 17.56 | |
| ZO-AdamBackbone=Llama-3.2 1B, Tuning=20-token prompt tuning2026.05 | 17.14 | |
| RAdaZOBackbone=Llama-3.2 1B, Tuning=20-token prompt tuning2026.05 | 16.9 | |
| MEAZOBackbone=Llama-3.2 1B, Tuning=20-token prompt tuning2026.05 | 16.85 | |
| BitDelta-col+distillRank=r16, BPW=1.00, Size (MB)=2.9, Time=+15 min2026.05 | 16.78 | |
| ZO-SGDBackbone=Qwen-3 0.6B, Tuning=20-token prompt tuning2026.05 | 16 | |
| MEAZOBackbone=Qwen-3 0.6B, Tuning=20-token prompt tuning2026.05 | 15.72 | |
| ZO-AdamBackbone=Qwen-3 0.6B, Tuning=20-token prompt tuning2026.05 | 15.62 | |
| PTQ-LoRDBARank=r16, BPW=2.0, Size (MB)=5.8, Time=+2 min2026.05 | 15.55 | |
| QAT FreezeRank=r16, BPW=1.5, Size (MB)=4.4, Time=+4h2026.05 | 15.54 | |
| PTQ-LoRDBARank=r16, BPW=1.5, Size (MB)=4.4, Time=+2 min2026.05 | 15.52 | |
| LoRAQuant ρ=0.9Rank=r16, BPW=1.71, Size (MB)=5.0, Time=+17 min2026.05 | 15.49 | |
| QAT FullRank=r16, BPW=2.0, Size (MB)=5.8, Time=+7h2026.05 | 15.49 | |
| LoRA FP16Rank=r16, BPW=16.0, Size (MB)=46.4, Time=–2026.05 | 15.38 | |
| QAT FreezeRank=r16, BPW=4.0, Size (MB)=11.6, Time=+4h2026.05 | 15.38 | |
| QAT FullRank=r16, BPW=4.0, Size (MB)=11.6, Time=+7h2026.05 | 15.35 | |
| QAT FullRank=r16, BPW=1.5, Size (MB)=4.4, Time=+7h2026.05 | 15.34 | |
| PTQ-LoRDBARank=r16, BPW=4.0, Size (MB)=11.6, Time=+2 min2026.05 | 15.19 | |
| QAT FreezeRank=r16, BPW=2.0, Size (MB)=5.8, Time=+4h2026.05 | 15.18 | |
| BaseBackbone=Qwen-3, Model Size=0.6B, Fine-tuning=None (Before tuning)2026.05 | 15.09 | |
| BaseBackbone=Qwen-3 0.6B, Tuning=20-token prompt tuning2026.05 | 15.09 | |
| QAT ScratchRank=r16, BPW=4.0, Size (MB)=11.6, Time=7h2026.05 | 15.07 | |
| QAT ScratchRank=r16, BPW=2.0, Size (MB)=5.8, Time=7h2026.05 | 15.05 | |
| LoRAQuant ρ=0.5Rank=r16, BPW=1.22, Size (MB)=3.6, Time=+8 min2026.05 | 15.02 | |
| QAT ScratchRank=r16, BPW=1.5, Size (MB)=4.4, Time=7h2026.05 | 14.72 | |
| QAT ScratchRank=r64, BPW=1.0, Size (MB)=11.6, Time=7h2026.05 | 14.54 | |
| BaseBackbone=Llama-3.2, Model Size=1B, Fine-tuning=None (Before tuning)2026.05 | 13.62 | |
| BaseBackbone=Llama-3.2 1B, Tuning=20-token prompt tuning2026.05 | 13.62 | |
| RAdaZOBackbone=Qwen-3 0.6B, Tuning=20-token prompt tuning2026.05 | 13.25 |