Language Modeling on LAMBADA (retention)
35.44PPLVeRA (CE + TMKL)
Evaluation Results
| Method | Links | |
|---|---|---|
| VeRA (CE + TMKL)Backbone=Qwen2.5-0.5B, Adapter=VeRA, Method=CE + TMKL2026.05 | 35.44 | |
| VeRA (CE)Backbone=Qwen2.5-0.5B, Adapter=VeRA, Method=CE2026.05 | 35.52 | |
| Base Qwen2.5-0.5BBackbone=Qwen2.5-0.5B, Adapter=None, Method=Base2026.05 | 35.68 | |
| PiSSA (CE + TMKL)Backbone=Qwen2.5-0.5B, Adapter=PiSSA, Method=CE + TMKL2026.05 | 48.48 | |
| AdaLoRA (CE + TMKL)Backbone=Qwen2.5-0.5B, Adapter=AdaLoRA, Method=CE + TMKL2026.05 | 58.93 | |
| AdaLoRA (CE)Backbone=Qwen2.5-0.5B, Adapter=AdaLoRA, Method=CE2026.05 | 60.54 | |
| PiSSA (CE)Backbone=Qwen2.5-0.5B, Adapter=PiSSA, Method=CE2026.05 | 77.18 |