Language Modeling on WT-103 (Retention Drift)
4Retention Drift (Δ%)CE + TMKL
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CE + TMKLBase Model=Qwen2.5-0.5B, Adapter Type=LoRA, Stability Hyperparameter (λ)=12026.05 | 4 | 89 | |
| LwF / Full-KLBase Model=Qwen2.5-0.5B, Adapter Type=LoRA, Stability Hyperparameter (λ)=12026.05 | 10 | 73 | |
| STABLEBase Model=Qwen2.5-0.5B, Adapter Type=LoRA2026.05 | 12 | 68 | |
| O-LoRABase Model=Qwen2.5-0.5B, Adapter Type=LoRA2026.05 | 22 | 41 | |
| EWCBase Model=Qwen2.5-0.5B, Adapter Type=LoRA2026.05 | 28 | 24 | |
| L2-SPBase Model=Qwen2.5-0.5B, Adapter Type=LoRA2026.05 | 32 | 14 | |
| LoRA + CE (no regularizer)Base Model=Qwen2.5-0.5B, Adapter Type=LoRA2026.05 | 37 | — |