Language Modeling on WikiText-103 (retention)
16.29PerplexityVeRA (CE)
Evaluation results
| Method | Links | |
|---|---|---|
| VeRA (CE)Backbone=Qwen2.5-0.5B, Adapter=VeRA, Method=CE2026.05 | 16.29 | |
| VeRA (CE + TMKL)Backbone=Qwen2.5-0.5B, Adapter=VeRA, Method=CE + TMKL2026.05 | 16.37 | |
| Base Qwen2.5-0.5BBackbone=Qwen2.5-0.5B, Adapter=None, Method=Base2026.05 | 16.46 | |
| PiSSA (CE + TMKL)Backbone=Qwen2.5-0.5B, Adapter=PiSSA, Method=CE + TMKL2026.05 | 24.16 | |
| AdaLoRA (CE + TMKL)Backbone=Qwen2.5-0.5B, Adapter=AdaLoRA, Method=CE + TMKL2026.05 | 27.45 | |
| AdaLoRA (CE)Backbone=Qwen2.5-0.5B, Adapter=AdaLoRA, Method=CE2026.05 | 28.4 | |
| PiSSA (CE)Backbone=Qwen2.5-0.5B, Adapter=PiSSA, Method=CE2026.05 | 38.86 |