Fine-tuning on WizardLM
0.661Evaluation LossRefLoRA
Evaluation results
| Method | Links | |
|---|---|---|
| RefLoRABackbone=Qwen-2.5-3B, Rank (r)=82026.05 | 0.661 | |
| DoRABackbone=Qwen-2.5-3B, Rank (r)=82026.05 | 0.662 | |
| BaLoRABackbone=Qwen-2.5-3B, Rank (r)=82026.05 | 0.662 | |
| LoRABackbone=Qwen-2.5-3B, Rank (r)=82026.05 | 0.663 | |
| LoRA-RITEBackbone=Qwen-2.5-3B, Rank (r)=82026.05 | 0.663 | |
| OLoRABackbone=Qwen-2.5-3B, Rank (r)=82026.05 | 0.666 | |
| LoROBackbone=Qwen-2.5-3B, Rank (r)=82026.05 | 0.689 |