Continual Learning on CoIN
-4.67Backward Transfer (BWT)LLaVA-DyMoE
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| LLaVA-DyMoEBackbone=LLaVA-v1.5-7B2026.03 | -4.67 | 76.25 | 53.86 | 95.8 | 48.4 | 52.35 | 9.25 | 58.3 | 62 | 57.03 | 57.7 | — | — | — | |
| DSCABackbone=PaliGemma-3B2026.04 | -9.37 | — | — | — | — | — | — | — | — | — | — | 49.96 | 11.04 | 76.48 | |
| DSCABackbone=PaliGemma-3B2026.04 | -9.37 | — | — | — | — | — | — | — | — | — | — | 49.96 | 11.04 | 76.48 | |
| IncMoELoRABackbone=LLaVA-v1.5-7B2026.03 | -16.67 | 68.43 | 50.31 | 68.42 | 47.97 | 39.46 | 4.56 | 57.31 | 60.95 | 49.68 | 49.5 | — | — | — | |
| O-LoRABackbone=LLaVA-v1.5-7B2026.03 | -17.54 | 75.61 | 49.98 | 78.24 | 44.18 | 30.7 | 4.66 | 55.51 | 57.37 | 49.53 | 46.65 | — | — | — | |
| PAMTraining Strategy=PAM2026.04 | -19.45 | — | — | — | — | — | — | — | — | — | — | 49.89 | 11.11 | 76.31 | |
| PAM2026.04 | -19.45 | — | — | — | — | — | — | — | — | — | — | 49.89 | 11.11 | 76.31 | |
| LWFBackbone=LLaVA-v1.5-7B2026.03 | -19.69 | 62.32 | 48.66 | 51.45 | 45.84 | 43.76 | 0.24 | 54.96 | 44.63 | 43.98 | 44.89 | — | — | — | |
| EWCBackbone=LLaVA-v1.5-7B2026.03 | -21.76 | 59.11 | 47.21 | 39.88 | 45.12 | 35.33 | 2.72 | 56.29 | 41.21 | 40.86 | 43.75 | — | — | — | |
| MoELoRABackbone=LLaVA-v1.5-7B2026.03 | -22.18 | 72.01 | 46.89 | 44.75 | 42.79 | 28.22 | 3.31 | 55.74 | 57.72 | 43.93 | 43.92 | — | — | — | |
| MagMaxTraining Strategy=MagMax2026.04 | -22.68 | — | — | — | — | — | — | — | — | — | — | 45.74 | 4.75 | 76.29 | |
| MagMax2026.04 | -22.68 | — | — | — | — | — | — | — | — | — | — | 45.74 | 4.75 | 76.29 | |
| LoRABackbone=LLaVA-v1.5-7B2026.03 | -23.12 | 52.56 | 48.12 | 39.27 | 44.47 | 37.46 | 1.22 | 56.1 | 55.11 | 41.79 | 43.99 | — | — | — | |
| IncLoRABackbone=LLaVA-v1.5-7B2026.03 | -23.21 | 73.33 | 44.32 | 54.59 | 44.07 | 25.93 | 4.49 | 54.91 | 58.55 | 45.02 | 43.12 | — | — | — | |
| O-LoRATraining Strategy=Orthogonal LoRA2026.04 | -32.08 | — | — | — | — | — | — | — | — | — | — | 46.53 | 9.45 | 73.27 | |
| LWFTraining Strategy=Learning without Forgetting2026.04 | -33.66 | — | — | — | — | — | — | — | — | — | — | 47.15 | 9.67 | 75.2 | |
| MoELoRATraining Strategy=Mixture-of-Experts LoRA2026.04 | -36.4 | — | — | — | — | — | — | — | — | — | — | 46.59 | 7.79 | 76.93 | |
| MoELoRA2026.04 | -36.4 | — | — | — | — | — | — | — | — | — | — | 46.59 | 7.79 | 76.93 | |
| FINE-TUNETraining Strategy=Fine-Tuning2026.04 | -39.51 | — | — | — | — | — | — | — | — | — | — | 43.36 | 7.71 | 76.29 | |
| I-LoRATraining Strategy=Incremental LoRA2026.04 | -40.95 | — | — | — | — | — | — | — | — | — | — | 42.11 | 7.89 | 76.24 | |
| INDEPENDENTTraining Strategy=Independent2026.04 | — | — | — | — | — | — | — | — | — | — | — | 76.46 | — | — | |
| INDEPENDENT2026.04 | — | — | — | — | — | — | — | — | — | — | — | 76.46 | — | — | |
| MULTITASKTraining Strategy=Multitask2026.04 | — | — | — | — | — | — | — | — | — | — | — | 73.93 | — | — | |
| MULTITASK2026.04 | — | — | — | — | — | — | — | — | — | — | — | 73.93 | — | — | |
| ZERO-SHOTTraining Strategy=Zero-Shot2026.04 | — | — | — | — | — | — | — | — | — | — | — | 24.74 | — | — | |
| ZERO-SHOT2026.04 | — | — | — | — | — | — | — | — | — | — | — | 24.74 | — | — |