Language Modeling on OpenR1
2.64Perplexity (PPL)PiSSA (CE)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| PiSSA (CE)Backbone=Phi-3.5-mini-instruct 3.8B, Adapter=PiSSA, Method=CE2026.05 | 2.64 | 0.86 | |
| PiSSA (CE + TMKL)Backbone=Phi-3.5-mini-instruct 3.8B, Adapter=PiSSA, Method=CE + TMKL2026.05 | 2.69 | 0.81 | |
| LoRA (CE + TMKL)Backbone=Phi-3.5-mini-instruct 3.8B, Adapter=LoRA, Method=CE + TMKL2026.05 | 2.71 | 0.79 | |
| DoRA (CE + TMKL)Backbone=Phi-3.5-mini-instruct 3.8B, Adapter=DoRA, Method=CE + TMKL2026.05 | 2.71 | 0.79 | |
| RandLoRA (CE + TMKL)Backbone=Phi-3.5-mini-instruct 3.8B, Adapter=RandLoRA, Method=CE + TMKL2026.05 | 2.71 | 0.79 | |
| LoRA (CE)Backbone=Phi-3.5-mini-instruct 3.8B, Adapter=LoRA, Method=CE2026.05 | 2.82 | 0.68 | |
| DoRA (CE)Backbone=Phi-3.5-mini-instruct 3.8B, Adapter=DoRA, Method=CE2026.05 | 2.82 | 0.68 | |
| RandLoRA (CE)Backbone=Phi-3.5-mini-instruct 3.8B, Adapter=RandLoRA, Method=CE2026.05 | 3 | 0.5 | |
| SineLoRA (CE + TMKL)Backbone=Phi-3.5-mini-instruct 3.8B, Adapter=SineLoRA, Method=CE + TMKL2026.05 | 3.21 | 0.29 | |
| SineLoRA (CE)Backbone=Phi-3.5-mini-instruct 3.8B, Adapter=SineLoRA, Method=CE2026.05 | 3.39 | 0.1 | |
| no adaptationBackbone=Phi-3.5-mini-instruct 3.8B, Adapter=None, Method=no adaptation2026.05 | 3.5 | — |