Continual Learning on TRACE
18.5BWT (%)SeqLoRA
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| SeqLoRABackbone=Llama-2-7B-Chat2026.05 | 18.5 | — | 34.3 | — | — | — | — | — | |
| SeqLoRABase Model=meta-llama / LLaMA-2-7B-Chat2025.06 | 18.5 | — | 34.3 | — | — | — | — | — | |
| SeqLoRABase Model=LLaMA-2-7B-Chat, Category=LoRA / Regularization / Gradient2026.05 | 18.5 | — | — | — | — | — | 34.3 | — | |
| SeqLoRAFoundation Model=LLaMA-2-7B-Chat2025.06 | 18.5 | — | 34.3 | — | — | — | — | 1,132 | |
| GEMBackbone=Gemma-2B-it2026.05 | 18.25 | — | 26.48 | — | — | — | — | — | |
| GEMBase Model=google / Gemma-2B-it2025.06 | 18.25 | — | 26.48 | — | — | — | — | — | |
| SeqLoRABackbone=Llama-3.2-1B-Instruct2026.05 | 17.03 | — | 29.73 | — | — | — | — | — | |
| SeqLoRABase Model=meta-llama / LLaMA-3-1B-Instruct2025.06 | 17.03 | — | 29.73 | — | — | — | — | — | |
| EWCBackbone=Gemma-2B-it2026.05 | 16.96 | — | 28.35 | — | — | — | — | — | |
| EWCBase Model=google / Gemma-2B-it2025.06 | 16.96 | — | 28.35 | — | — | — | — | — | |
| L2PBackbone=Gemma-2B-it2026.05 | 15.77 | — | 31.14 | — | — | — | — | — | |
| L2PBase Model=google / Gemma-2B-it2025.06 | 15.77 | — | 31.14 | — | — | — | — | — | |
| EWCModel=Vicuna-7B2026.05 | 15.57 | 41.88 | — | — | — | — | — | — | |
| SeqLoRABackbone=Gemma-2B-it2026.05 | 15.28 | — | 31.89 | — | — | — | — | — | |
| SeqLoRABase Model=google / Gemma-2B-it2025.06 | 15.28 | — | 31.89 | — | — | — | — | — | |
| OGDBase Model=meta-llama / LLaMA-3-1B-Instruct2025.06 | 15.2 | — | 30.12 | — | — | — | — | — | |
| DualPromptBackbone=Gemma-2B-it2026.05 | 14.25 | — | 32.42 | — | — | — | — | — | |
| DualPromptBase Model=google / Gemma-2B-it2025.06 | 14.25 | — | 32.42 | — | — | — | — | — | |
| HiDeLoRABackbone=Gemma-2B-it2026.05 | 13.66 | — | 33.25 | — | — | — | — | — | |
| HiDeLoRABase Model=google / Gemma-2B-it2025.06 | 13.66 | — | 33.25 | — | — | — | — | — | |
| L2PBackbone=Llama-3.2-1B-Instruct2026.05 | 13.57 | — | 29.38 | — | — | — | — | — | |
| L2PBase Model=meta-llama / LLaMA-3-1B-Instruct2025.06 | 13.57 | — | 29.38 | — | — | — | — | — | |
| O-LoRABackbone=Llama-3.2-1B-Instruct2026.05 | 12.89 | — | 32.94 | — | — | — | — | — | |
| O-LoRABase Model=meta-llama / LLaMA-3-1B-Instruct2025.06 | 12.89 | — | 32.94 | — | — | — | — | — | |
| O-LoRABackbone=Gemma-2B-it2026.05 | 12.36 | — | 33.73 | — | — | — | — | — | |
| HiDeLoRABackbone=Llama-3.2-1B-Instruct2026.05 | 12.36 | — | 33.73 | — | — | — | — | — | |
| O-LoRABase Model=google / Gemma-2B-it2025.06 | 12.36 | — | 33.73 | — | — | — | — | — | |
| HiDeLoRABase Model=meta-llama / LLaMA-3-1B-Instruct2025.06 | 12.36 | — | 33.73 | — | — | — | — | — | |
| OGDBase Model=google / Gemma-2B-it2025.06 | 12.27 | — | 32.85 | — | — | — | — | — | |
| EWCBackbone=Llama-3.2-1B-Instruct2026.05 | 11.62 | — | 31.96 | — | — | — | — | — | |
| EWCBase Model=meta-llama / LLaMA-3-1B-Instruct2025.06 | 11.62 | — | 31.96 | — | — | — | — | — | |
| LFPT5Base Model=LLaMA-2-7B-Chat, Category=Prompt-based2026.05 | 11.43 | — | — | — | — | — | 38.67 | — | |
| SeqLoRAModel=Mistral-7B2026.05 | 11.41 | 46.94 | — | — | — | — | — | — | |
| DualPromptBackbone=Llama-3.2-1B-Instruct2026.05 | 11.34 | — | 30.76 | — | — | — | — | — | |
| DualPromptBase Model=meta-llama / LLaMA-3-1B-Instruct2025.06 | 11.34 | — | 30.76 | — | — | — | — | — | |
| GEMBackbone=Llama-3.2-1B-Instruct2026.05 | 10.74 | — | 32.19 | — | — | — | — | — | |
| GEMBase Model=meta-llama / LLaMA-3-1B-Instruct2025.06 | 10.74 | — | 32.19 | — | — | — | — | — | |
| DER++Model=Vicuna-7B, memory_buffer_size=1%2026.05 | 9.04 | 49.01 | — | — | — | — | — | — | |
| TreeLoRABackbone=Gemma-2B-it2026.05 | 8.5 | — | 33.41 | — | — | — | — | — | |
| TreeLoRABase Model=google / Gemma-2B-it2025.06 | 8.5 | — | 33.41 | — | — | — | — | — | |
| L2PBackbone=LLaMA-2-7B-Chat2026.05 | 8.25 | — | 36.23 | — | — | — | — | — | |
| L2PBackbone=Llama-2-7B-Chat2026.05 | 8.25 | — | 36.23 | — | — | — | — | — | |
| L2PBase Model=meta-llama / LLaMA-2-7B-Chat2025.06 | 8.25 | — | 36.23 | — | — | — | — | — | |
| L2PBase Model=LLaMA-2-7B-Chat, Category=Prompt-based2026.05 | 8.25 | — | — | — | — | — | 36.23 | — | |
| L2PFoundation Model=LLaMA-2-7B-Chat2025.06 | 8.25 | — | 36.23 | — | — | — | — | 899 | |
| O-LoRAModel=Mistral-7B2026.05 | 8.13 | 52.02 | — | — | — | — | — | — | |
| OGDBase Model=meta-llama / LLaMA-2-7B-Chat2025.06 | 8.06 | — | 42.09 | — | — | — | — | — | |
| OGDBase Model=LLaMA-2-7B-Chat, Category=LoRA / Regularization / Gradient2026.05 | 8.06 | — | — | — | — | — | 42.09 | — | |
| OGDFoundation Model=LLaMA-2-7B-Chat2025.06 | 8.06 | — | 42.09 | — | — | — | — | 6,416 | |
| DualPromptBackbone=LLaMA-2-7B-Chat2026.05 | 8.03 | — | 37.69 | — | — | — | — | — | |
| DualPromptBackbone=Llama-2-7B-Chat2026.05 | 8.03 | — | 37.69 | — | — | — | — | — | |
| DualPromptBase Model=meta-llama / LLaMA-2-7B-Chat2025.06 | 8.03 | — | 37.69 | — | — | — | — | — | |
| DualPromptBase Model=LLaMA-2-7B-Chat, Category=Prompt-based2026.05 | 8.03 | — | — | — | — | — | 37.69 | — | |
| DualPromptFoundation Model=LLaMA-2-7B-Chat2025.06 | 8.03 | — | 37.69 | — | — | — | — | 912 | |
| TreeLoRABackbone=Llama-3.2-1B-Instruct2026.05 | 7.36 | — | 36.14 | — | — | — | — | — | |
| TreeLoRABase Model=meta-llama / LLaMA-3-1B-Instruct2025.06 | 7.36 | — | 36.14 | — | — | — | — | — | |
| O-LoRABackbone=LLaMA-2-7B-Chat2026.05 | 7.16 | — | 42.78 | — | — | — | — | — | |
| O-LoRABackbone=Llama-2-7B-Chat2026.05 | 7.16 | — | 42.78 | — | — | — | — | — | |
| O-LoRABase Model=meta-llama / LLaMA-2-7B-Chat2025.06 | 7.16 | — | 42.78 | — | — | — | — | — | |
| O-LoRABase Model=LLaMA-2-7B-Chat, Category=LoRA / Regularization / Gradient2026.05 | 7.16 | — | — | — | — | — | 42.78 | — | |
| O-LoRAFoundation Model=LLaMA-2-7B-Chat2025.06 | 7.16 | — | 42.78 | — | — | — | — | 1,293 | |
| HiDeLoRABackbone=LLaMA-2-7B-Chat2026.05 | 7.12 | — | 41.6 | — | — | — | — | — | |
| HiDeLoRABackbone=Llama-2-7B-Chat2026.05 | 7.12 | — | 41.6 | — | — | — | — | — | |
| HiDeLoRABase Model=meta-llama / LLaMA-2-7B-Chat2025.06 | 7.12 | — | 41.6 | — | — | — | — | — | |
| HiDe-PETBase Model=LLaMA-2-7B-Chat, Category=LoRA / Regularization / Gradient2026.05 | 7.12 | — | — | — | — | — | 41.6 | — | |
| HiDeLoRAFoundation Model=LLaMA-2-7B-Chat2025.06 | 7.12 | — | 41.6 | — | — | — | — | 1,286 | |
| GEMBackbone=LLaMA-2-7B-Chat2026.05 | 6.77 | — | 40.08 | — | — | — | — | — | |
| GEMBackbone=Llama-2-7B-Chat2026.05 | 6.77 | — | 40.08 | — | — | — | — | — | |
| GEMBase Model=meta-llama / LLaMA-2-7B-Chat2025.06 | 6.77 | — | 40.08 | — | — | — | — | — | |
| GEMBase Model=LLaMA-2-7B-Chat, Category=LoRA / Regularization / Gradient2026.05 | 6.77 | — | — | — | — | — | 40.08 | — | |
| GEMFoundation Model=LLaMA-2-7B-Chat2025.06 | 6.77 | — | 40.08 | — | — | — | — | 7,385 | |
| LwFBase Model=LLaMA-2-7B-Chat, Category=LoRA / Regularization / Gradient2026.05 | 6.5 | — | — | — | — | — | 41.86 | — | |
| O-LoRAModel=Vicuna-7B2026.05 | 6.27 | 43.42 | — | — | — | — | — | — | |
| HiDe-PETModel=Mistral-7B2026.05 | 6.25 | 51.81 | — | — | — | — | — | — | |
| DualPromptModel=Mistral-7B2026.05 | 6.13 | 51.14 | — | — | — | — | — | — | |
| GEMModel=Mistral-7B2026.05 | 6.01 | 52.32 | — | — | — | — | — | — | |
| EWCModel=Mistral-7B2026.05 | 5.98 | 52.45 | — | — | — | — | — | — | |
| EWCBackbone=LLaMA-2-7B-Chat2026.05 | 5.97 | — | 42.36 | — | — | — | — | — | |
| EWCBackbone=Llama-2-7B-Chat2026.05 | 5.97 | — | 42.36 | — | — | — | — | — | |
| EWCBase Model=meta-llama / LLaMA-2-7B-Chat2025.06 | 5.97 | — | 42.36 | — | — | — | — | — | |
| EWCBase Model=LLaMA-2-7B-Chat, Category=LoRA / Regularization / Gradient2026.05 | 5.97 | — | — | — | — | — | 42.36 | — | |
| EWCFoundation Model=LLaMA-2-7B-Chat2025.06 | 5.97 | — | 42.36 | — | — | — | — | 50,283 | |
| PPBase Model=LLaMA-2-7B-Chat, Category=Prompt-based2026.05 | 5.79 | — | — | — | — | — | 29.41 | — | |
| LwFModel=Vicuna-7B2026.05 | 5.54 | 41.19 | — | — | — | — | — | — | |
| SAPTBackbone=LLaMA-2-7B-Chat, generative replay=false2026.05 | 5.49 | — | 42.93 | — | — | — | — | — | |
| SAPTFoundation Model=LLaMA-2-7B-Chat2025.06 | 5.49 | — | 42.93 | — | — | — | — | 1,205 | |
| L2PModel=Mistral-7B2026.05 | 5.34 | 49.32 | — | — | — | — | — | — | |
| TASLBackbone=LLaMA-2-7B-Chat2026.05 | 4.58 | — | 43.19 | — | — | — | — | — | |
| TASLFoundation Model=LLaMA-2-7B-Chat2025.06 | 4.58 | — | 43.19 | — | — | — | — | 1,185 | |
| TreeLoRAModel=Mistral-7B2026.05 | 3.77 | 54.77 | — | — | — | — | — | — | |
| TreeLoRABackbone=LLaMA-2-7B-Chat2026.05 | 3.46 | — | 43.52 | — | — | — | — | — | |
| TreeLoRABackbone=Llama-2-7B-Chat2026.05 | 3.46 | — | 43.52 | — | — | — | — | — | |
| TreeLoRABase Model=meta-llama / LLaMA-2-7B-Chat2025.06 | 3.46 | — | 43.52 | — | — | — | — | — | |
| TreeLoRABase Model=LLaMA-2-7B-Chat, Category=LoRA / Regularization / Gradient2026.05 | 3.46 | — | — | — | — | — | 43.52 | — | |
| TreeLoRAFoundation Model=LLaMA-2-7B-Chat2025.06 | 3.46 | — | 43.52 | — | — | — | — | 485 | |
| MoRAMBase Model=meta-llama / LLaMA-3-1B-Instruct2025.06 | 3.12 | — | 37.77 | — | — | — | — | — | |
| MoRAMBase Model=google / Gemma-2B-it2025.06 | 2.74 | — | 36.27 | — | — | — | — | — | |
| SEEKRModel=Vicuna-7B, memory_buffer_size=1%2026.05 | 2.64 | 55.78 | — | — | — | — | — | — | |
| CRAFTBackbone=Gemma-2B-it2026.05 | 2.47 | — | 35.18 | — | — | — | — | — | |
| CRAFTBackbone=LLaMA-2-7B-Chat2026.05 | 2.29 | — | 43.69 | — | — | — | — | — |