List your company
ResearchFormerly WizWand

Instruction Tuning on MT-Bench

6.55ScoreLR-LoRA
4.09564.73285.376.0072May 9, 2026May 13, 2026May 17, 2026May 21, 2026May 25, 2026May 29, 2026Jun 3, 2026
Updated 5mo ago

Evaluation results

MethodLinks
LR-LoRABackbone=LLaMA3-8B, r=322026.066.55
RandLoRABackbone=LLaMA3-8B, r=322026.066.31
SineLoRABackbone=LLaMA3-8B, r=322026.066.22
LoRABackbone=LLaMA3-8B, r=322026.066.12
S2FTModel=LLaMA2-13B, # Trainable Parameters=0.08M2026.055.89
FourierFTModel=LLaMA2-13B, # Trainable Parameters=0.08M2026.055.82
LoRAModel=LLaMA2-13B, # Trainable Parameters=52.4M, Updating layers other than lm_head=false2026.055.8
LoRAModel=LLaMA2-13B, # Trainable Parameters=250.3M, Updating layers other than lm_head=true2026.055.78
S2FTModel=LLaMA2-7B, # Trainable Parameters=0.064M2026.055.21
LoRAModel=LLaMA2-7B, # Trainable Parameters=33.5M, Updating layers other than lm_head=false2026.055.2
LoRAModel=LLaMA2-7B, # Trainable Parameters=159.9M, Updating layers other than lm_head=true2026.055.19
FourierFTModel=LLaMA2-7B, # Trainable Parameters=0.064M2026.055.18
LR-LoRABackbone=Qwen2-0.5B, r=322026.064.61
RandLoRABackbone=Qwen2-0.5B, r=322026.064.33
SineLoRABackbone=Qwen2-0.5B, r=322026.064.26
LoRABackbone=Qwen2-0.5B, r=322026.064.19