Language Modeling on OpenR1-Math
2.86Perplexity (PPL)LwF / Full-KL
Evaluation results
| Method | Links | |
|---|---|---|
| LwF / Full-KLBase Model=Qwen2.5-0.5B, Adapter Type=LoRA, Stability Hyperparameter (λ)=12026.05 | 2.86 | |
| CE + TMKLBase Model=Qwen2.5-0.5B, Adapter Type=LoRA, Stability Hyperparameter (λ)=12026.05 | 2.87 | |
| STABLEBase Model=Qwen2.5-0.5B, Adapter Type=LoRA2026.05 | 2.9 | |
| O-LoRABase Model=Qwen2.5-0.5B, Adapter Type=LoRA2026.05 | 2.98 | |
| PiSSA (CE + TMKL)Backbone=Qwen2.5-0.5B, Adapter=PiSSA, Method=CE + TMKL2026.05 | 2.99 | |
| EWCBase Model=Qwen2.5-0.5B, Adapter Type=LoRA2026.05 | 3.01 | |
| L2-SPBase Model=Qwen2.5-0.5B, Adapter Type=LoRA2026.05 | 3.03 | |
| LoRA + CE (no regularizer)Base Model=Qwen2.5-0.5B, Adapter Type=LoRA2026.05 | 3.05 | |
| VeRA (CE)Backbone=Qwen2.5-0.5B, Adapter=VeRA, Method=CE2026.05 | 3.15 | |
| VeRA (CE + TMKL)Backbone=Qwen2.5-0.5B, Adapter=VeRA, Method=CE + TMKL2026.05 | 3.16 | |
| PiSSA (CE)Backbone=Qwen2.5-0.5B, Adapter=PiSSA, Method=CE2026.05 | 3.2 | |
| Base Qwen2.5-0.5BBase Model=Qwen2.5-0.5B, Adapter Type=None2026.05 | 3.21 | |
| Base Qwen2.5-0.5BBackbone=Qwen2.5-0.5B, Adapter=None, Method=Base2026.05 | 3.21 | |
| AdaLoRA (CE)Backbone=Qwen2.5-0.5B, Adapter=AdaLoRA, Method=CE2026.05 | 4.96 | |
| AdaLoRA (CE + TMKL)Backbone=Qwen2.5-0.5B, Adapter=AdaLoRA, Method=CE + TMKL2026.05 | 5.08 |