Instruction Following on Stanford Alpaca (val)
1.05Validation LossLoRA
Evaluation Results
| Method | Links | |
|---|---|---|
| LoRABackbone=LLaMA-2 13B (4-bit), Rank=r=1, # Params=819K2026.04 | 1.05 | |
| SOLARBackbone=LLaMA-2 13B (4-bit), Rank=r = 1(1→0.3), # Params=51K (94% ↓)2026.04 | 1.05 | |
| NOLABackbone=LLaMA-2 13B (4-bit), Rank=1000, # Params=140K2026.04 | 1.29 | |
| LoRABackbone=LLaMA-3.2 1B, Rank=r=8, # Params=852K2026.04 | 1.51 | |
| SOLARBackbone=LLaMA-3.2 1B, Rank=r = 8(4→1.2), # Params=81K (90% ↓)2026.04 | 1.52 | |
| NOLABackbone=LLaMA-3.2 1B, Rank=1000, # Params=64K2026.04 | 1.87 |