Instruction Tuning on MT-Bench
6.55ScoreLR-LoRA
Evaluation Results
| Method | Links | |
|---|---|---|
| LR-LoRABackbone=LLaMA3-8B, r=322026.06 | 6.55 | |
| RandLoRABackbone=LLaMA3-8B, r=322026.06 | 6.31 | |
| SineLoRABackbone=LLaMA3-8B, r=322026.06 | 6.22 | |
| LoRABackbone=LLaMA3-8B, r=322026.06 | 6.12 | |
| S2FTModel=LLaMA2-13B, # Trainable Parameters=0.08M2026.05 | 5.89 | |
| FourierFTModel=LLaMA2-13B, # Trainable Parameters=0.08M2026.05 | 5.82 | |
| LoRAModel=LLaMA2-13B, # Trainable Parameters=52.4M, Updating layers other than lm_head=false2026.05 | 5.8 | |
| LoRAModel=LLaMA2-13B, # Trainable Parameters=250.3M, Updating layers other than lm_head=true2026.05 | 5.78 | |
| S2FTModel=LLaMA2-7B, # Trainable Parameters=0.064M2026.05 | 5.21 | |
| LoRAModel=LLaMA2-7B, # Trainable Parameters=33.5M, Updating layers other than lm_head=false2026.05 | 5.2 | |
| LoRAModel=LLaMA2-7B, # Trainable Parameters=159.9M, Updating layers other than lm_head=true2026.05 | 5.19 | |
| FourierFTModel=LLaMA2-7B, # Trainable Parameters=0.064M2026.05 | 5.18 | |
| LR-LoRABackbone=Qwen2-0.5B, r=322026.06 | 4.61 | |
| RandLoRABackbone=Qwen2-0.5B, r=322026.06 | 4.33 | |
| SineLoRABackbone=Qwen2-0.5B, r=322026.06 | 4.26 | |
| LoRABackbone=Qwen2-0.5B, r=322026.06 | 4.19 |