Fine-tuning on OpenOrca
0.773Evaluation LossRefLoRA
Evaluation results
| Method | Links | |
|---|---|---|
| RefLoRABackbone=Qwen-2.5-3B, Rank (r)=82026.05 | 0.773 | |
| BaLoRABackbone=Qwen-2.5-3B, Rank (r)=82026.05 | 0.773 | |
| LoRABackbone=Qwen-2.5-3B, Rank (r)=82026.05 | 0.774 | |
| DoRABackbone=Qwen-2.5-3B, Rank (r)=82026.05 | 0.776 | |
| LoRA-RITEBackbone=Qwen-2.5-3B, Rank (r)=82026.05 | 0.776 | |
| OLoRABackbone=Qwen-2.5-3B, Rank (r)=82026.05 | 0.782 | |
| LoROBackbone=Qwen-2.5-3B, Rank (r)=82026.05 | 0.859 |