Knowledge Retention on Source domain
65.9Source Domain AccuracyLaLoRA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| LaLoRAlambda=5 * 10^32025.12 | 65.9 | — | 7.1 | |
| L2 Regularizationstrength=0.52025.12 | 65.7 | — | 6.9 | |
| L2 Regularizationstrength=0.12025.12 | 65.7 | — | 6.9 | |
| L2 Regularizationstrength=0.052025.12 | 65.7 | — | 6.9 | |
| MIGUt=0.952025.12 | 65.3 | — | 6.5 | |
| LaLoRAlambda=10^32025.12 | 64.9 | — | 6.1 | |
| L2 Regularizationstrength=10^-32025.12 | 64.4 | — | 5.7 | |
| LaLoRAlambda=2.5 * 10^22025.12 | 64 | — | 5.2 | |
| LaLoRAlambda=1.5 * 10^22025.12 | 63.6 | — | 4.8 | |
| MIGUt=0.852025.12 | 63.4 | — | 4.6 | |
| LaLoRAlambda=10^22025.12 | 63.2 | — | 4.4 | |
| MIGUt=0.92025.12 | 63.2 | — | 4.4 | |
| MIGUt=0.82025.12 | 63.1 | — | 4.3 | |
| MIGUt=0.72025.12 | 62.4 | — | 3.6 | |
| LaLoRAlambda=102025.12 | 61.8 | — | 3 | |
| MIGUt=0.6, variant=a2025.12 | 61.8 | — | 3 | |
| MIGUt=0.6, variant=b2025.12 | 61.7 | — | 2.9 | |
| L2 Regularizationstrength=10^-42025.12 | 61.2 | — | 2.4 | |
| LaLoRAlambda=12025.12 | 60.4 | — | 1.6 | |
| L2 Regularizationstrength=10^-52025.12 | 60 | — | 1.2 | |
| MiLoRA2025.12 | 59.9 | — | 1.1 | |
| L2 Regularizationstrength=10^-62025.12 | 59.1 | — | 0.4 | |
| LoRAlambda=02025.12 | 58.8 | — | 0 | |
| Baseline (LoRA)2025.12 | — | 0 | — | |
| L2-regularizerHyperparameter priority=stability2025.12 | — | 4.8 | — | |
| L2-regularizerHyperparameter priority=plasticity2025.12 | — | 2.4 | — | |
| LA-LoRAHyperparameter priority=stability2025.12 | — | 4 | — | |
| LA-LoRAHyperparameter priority=plasticity2025.12 | — | 2.6 | — | |
| MIGUHyperparameter priority=stability2025.12 | — | 2.9 | — | |
| MIGUHyperparameter priority=plasticity2025.12 | — | 2.9 | — | |
| MiLoRA2025.12 | — | 1.4 | — |