Fine-tuning on Alpaca
1.35Evaluation LossRefLoRA
Evaluation results
| Method | Links | |
|---|---|---|
| RefLoRABackbone=Qwen-2.5-3B, Rank (r)=82026.05 | 1.35 | |
| BaLoRABackbone=Qwen-2.5-3B, Rank (r)=82026.05 | 1.35 | |
| LoRABackbone=Qwen-2.5-3B, Rank (r)=82026.05 | 1.352 | |
| DoRABackbone=Qwen-2.5-3B, Rank (r)=82026.05 | 1.352 | |
| LoRA-RITEBackbone=Qwen-2.5-3B, Rank (r)=82026.05 | 1.353 | |
| OLoRABackbone=Qwen-2.5-3B, Rank (r)=82026.05 | 1.36 | |
| LoROBackbone=Qwen-2.5-3B, Rank (r)=82026.05 | 1.504 |