Continual Learning on MMLU, ScienceQA, and GSM8K Sequence
62.1MMLU AccuracyNeuroLoRA-Seq
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| NeuroLoRA-SeqBackbone=Llama-3-8B, Training strategy=Sequential fine-tuning2026.03 | 62.1 | 92 | 60.4 | 71.5 | 2.6 | |
| FlyLoRA-SeqBackbone=Llama-3-8B, Training strategy=Sequential fine-tuning2026.03 | 59.7 | 90.1 | 57.6 | 69.1 | 4.3 | |
| O-LoRABackbone=Llama-3-8B, Training strategy=Sequential fine-tuning2026.03 | 58.2 | 89.4 | 55.6 | 67.7 | 5.8 | |
| EWC+LoRABackbone=Llama-3-8B, Training strategy=Sequential fine-tuning2026.03 | 56.1 | 87.5 | 54.8 | 66.1 | 7.5 | |
| LoRA-SeqBackbone=Llama-3-8B, Training strategy=Sequential fine-tuning2026.03 | 51.8 | 84.2 | 55.4 | 63.8 | 11.3 |