Fine-tuning on OpenHermes
0.706Evaluation LossRefLoRA
Evaluation results
| Method | Links | |
|---|---|---|
| RefLoRABackbone=Qwen-2.5-3B, Rank (r)=82026.05 | 0.706 | |
| LoRABackbone=Qwen-2.5-3B, Rank (r)=82026.05 | 0.707 | |
| DoRABackbone=Qwen-2.5-3B, Rank (r)=82026.05 | 0.707 | |
| LoRA-RITEBackbone=Qwen-2.5-3B, Rank (r)=82026.05 | 0.707 | |
| BaLoRABackbone=Qwen-2.5-3B, Rank (r)=82026.05 | 0.707 | |
| OLoRABackbone=Qwen-2.5-3B, Rank (r)=82026.05 | 0.712 | |
| LoROBackbone=Qwen-2.5-3B, Rank (r)=82026.05 | 0.75 |