Supervised Fine-tuning on SFT (evaluation)
3.13SFT Evaluation LossBHyT
Evaluation results
| Method | Links | |
|---|---|---|
| BHyTBackbone=Llama-3B, Fine-tuning=LoRA-based SFT2025.12 | 3.13 | |
| LNSBackbone=Llama-3B, Fine-tuning=LoRA-based SFT2025.12 | 3.157 | |
| Peri-LNBackbone=Llama-3B, Fine-tuning=LoRA-based SFT2025.12 | 3.178 | |
| RMSNormBackbone=Llama-3B, Fine-tuning=LoRA-based SFT2025.12 | 3.217 | |
| DyTBackbone=Llama-3B, Fine-tuning=LoRA-based SFT2025.12 | 3.971 |