Multi-Task Incremental Learning on MTIL Order II
89.2Average AccFine-tune
Evaluation results
| Method | Links | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Fine-tuneProtocol=Upper-bound2026.05 | 89.2 | — | — | — | — | 89.6 | 92.7 | 99.6 | 94.7 | 97.5 | 81.8 | 62 | 95.1 | 79.5 | 98.9 | 89.6 | |
| OursProtocol=Last2026.05 | 88.4 | — | 88.4 | — | — | 89 | 92.6 | 99.6 | 92.7 | 96.7 | 82.7 | 55.1 | 97.5 | 79 | 98.8 | 88.7 | |
| LEBAProtocol=Last2026.05 | 86.4 | — | 86.4 | — | — | 86.2 | 88.9 | 99.2 | 93 | 96.5 | 79.2 | 50.1 | 95.2 | 78.2 | 95.9 | 88.1 | |
| MoE-AdaptersProtocol=Last2026.05 | 84.1 | — | 84.1 | — | — | 84.1 | 88.5 | 94 | 91.8 | 94.1 | 77.8 | 50.4 | 93.3 | 77.1 | 87.7 | 86.6 | |
| ZSCLProtocol=Last2026.05 | 83.4 | — | 83.4 | — | — | 78.2 | 91.1 | 97.6 | 92.5 | 87.4 | 78.2 | 45 | 92.3 | 72.7 | 96.2 | 86.3 | |
| KeepLoRA++clsArch. Kept=✗, w/o Extra Data=✓2026.06 | 77 | 65.9 | 88.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OursProtocol=Avg.2026.05 | 76.8 | — | — | — | — | 89 | 92.3 | 92.2 | 91.7 | 87.4 | 75 | 38.3 | 91.8 | 54 | 63 | 70.1 | |
| iGSP2026.05 | 76.5 | 65.8 | 87.1 | 0.4 | — | — | — | — | — | — | — | — | — | — | — | — | |
| OursExtra data=×, Train Params.=30.8 M2026.04 | 76.4 | 65.7 | 88.1 | — | 76.7 | — | — | — | — | — | — | — | — | — | — | — | |
| Ours†Extra data=×, Train Params.=4.6 M2026.04 | 76 | 65.3 | 87.6 | — | 76.2 | — | — | — | — | — | — | — | — | — | — | — | |
| ConDUEvaluation Protocol=Fine-Tuning2025.03 | 75.9 | 66.5 | 85.6 | 1.1 | — | — | — | — | — | — | — | — | — | — | — | — | |
| KeepLoRA++Arch. Kept=✓, w/o Extra Data=✓2026.06 | 75.9 | 65.3 | 86 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GIFTBackbone=ViT-B/162025.03 | 75.7 | 65.9 | 85.3 | 8.1 | — | — | — | — | — | — | — | — | — | — | — | — | |
| GIFTExtra data=✓, Train Params.=149.6 M2026.04 | 75.7 | 65.9 | 85.3 | — | 75.6 | — | — | — | — | — | — | — | — | — | — | — | |
| LEBAProtocol=Avg.2026.05 | 75.6 | — | — | — | — | 86 | 88.9 | 92.1 | 91.9 | 87.2 | 72.8 | 33.9 | 90.9 | 54.7 | 62.7 | 70.1 | |
| MoE-Adapter++2026.05 | 75.1 | 64.8 | 85.6 | -0.6 | — | — | — | — | — | — | — | — | — | — | — | — | |
| IAPArch. Kept=✗, w/o Extra Data=✓2026.06 | 75.1 | 64.9 | 85.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MulKI2025.03 | 75 | 65.6 | 84.2 | 0.2 | — | — | — | — | — | — | — | — | — | — | — | — | |
| MoE-AdapterBackbone=ViT-B/162025.03 | 74.7 | 64.3 | 84.1 | 6.9 | — | — | — | — | — | — | — | — | — | — | — | — | |
| MoE2025.03 | 74.7 | 64.3 | 84.1 | -1.1 | — | — | — | — | — | — | — | — | — | — | — | — | |
| MoE-AdapterExtra data=×, Train Params.=59.6 M2026.04 | 74.7 | 64.3 | 84.1 | — | 74.4 | — | — | — | — | — | — | — | — | — | — | — | |
| MoE-AdaptersProtocol=Avg.2026.05 | 74.7 | — | — | — | — | 84.9 | 89.9 | 89.3 | 91.4 | 86.2 | 72.2 | 33.4 | 89.4 | 53.3 | 61.4 | 69.9 | |
| MoE-Adapter2026.05 | 74.7 | 64.3 | 84.1 | -1.1 | — | — | — | — | — | — | — | — | — | — | — | — | |
| MoE-AdaptersArch. Kept=✗, w/o Extra Data=✗2026.06 | 74.7 | 64.3 | 85 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ZSCLArch. Kept=✓, w/o Extra Data=✗2026.06 | 74.6 | 64.1 | 83.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ZSCLBackbone=ViT-B/162025.03 | 74.5 | 64.2 | 83.4 | 6.2 | — | — | — | — | — | — | — | — | — | — | — | — | |
| ZSCL2025.03 | 74.5 | 64.2 | 83.4 | -1.2 | — | — | — | — | — | — | — | — | — | — | — | — | |
| ZSCLExtra data=✓, Train Params.=149.6 M2026.04 | 74.5 | 64.2 | 83.4 | — | 74 | — | — | — | — | — | — | — | — | — | — | — | |
| DIKIExtra data=×, Train Params.=1.8 M2026.04 | 74.5 | 64.4 | 85.5 | — | 74.8 | — | — | — | — | — | — | — | — | — | — | — | |
| ZSCLProtocol=Avg.2026.05 | 74.5 | — | — | — | — | 81.7 | 91.3 | 91.1 | 91 | 82.9 | 72.5 | 33.6 | 89.7 | 53.3 | 62.8 | 69.9 | |
| ZSCL2026.05 | 74.5 | 64.1 | 83.4 | -1.3 | — | — | — | — | — | — | — | — | — | — | — | — | |
| DIKI2026.05 | 74.5 | 64.4 | 85.5 | -1 | — | — | — | — | — | — | — | — | — | — | — | — | |
| InfLoRAArch. Kept=✓, w/o Extra Data=✓2026.06 | 74.5 | 64 | 84.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DIKIArch. Kept=✗, w/o Extra Data=✓2026.06 | 74.5 | 64.4 | 85.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| O-LoRAArch. Kept=✓, w/o Extra Data=✓2026.06 | 72.9 | 63.4 | 80.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SD-LoRAArch. Kept=✓, w/o Extra Data=✓2026.06 | 72.6 | 63.7 | 81.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| WiSE-FTProtocol=Last2026.05 | 72.2 | — | 72.2 | — | — | 35.6 | 76.9 | 99.5 | 89.1 | 62.1 | 71.8 | 27.8 | 90.8 | 67 | 85.6 | 87.6 | |
| LwFProtocol=Last2026.05 | 71.9 | — | 71.9 | — | — | 34.6 | 69.6 | 99.3 | 88.7 | 61.1 | 72.5 | 32.5 | 88.1 | 65.6 | 90.9 | 87.9 | |
| iCaRLProtocol=Last2026.05 | 71.6 | — | 71.6 | — | — | 46 | 81.5 | 91.3 | 82.8 | 66.5 | 72.2 | 16.3 | 91.6 | 68.1 | 83.2 | 87.8 | |
| iGSP2026.05 | 70.8 | 65.8 | 76.6 | 0.4 | — | — | — | — | — | — | — | — | — | — | — | — | |
| MoE-Adapter++2026.05 | 70.4 | 65.2 | 76.3 | -0.2 | — | — | — | — | — | — | — | — | — | — | — | — | |
| MoE-Adapter2026.05 | 69.5 | 64.7 | 75.7 | -0.7 | — | — | — | — | — | — | — | — | — | — | — | — | |
| l2 baselineBackbone=ViT-B/162025.03 | 68.8 | 60.6 | 77.2 | 0 | — | — | — | — | — | — | — | — | — | — | — | — | |
| ZSCL2026.05 | 67.6 | 62.8 | 71.8 | -2.6 | — | — | — | — | — | — | — | — | — | — | — | — | |
| DualPrompt2026.05 | 67.5 | 52.1 | 82.8 | -13.3 | — | — | — | — | — | — | — | — | — | — | — | — | |
| DualPromptArch. Kept=✗, w/o Extra Data=✓2026.06 | 67.5 | 52.1 | 82.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Zero-shotProtocol=Upper-bound2026.05 | 65.4 | — | — | — | — | 64.7 | 88.5 | 59.4 | 89 | 71 | 65.2 | 24.3 | 88.4 | 44.6 | 54.9 | 68.2 | |
| OursProtocol=Transfer2026.05 | 65.4 | 65.4 | — | — | — | — | 88.5 | 58.9 | 89.1 | 71 | 65.9 | 24.4 | 88.5 | 44.7 | 55 | 68.2 | |
| Zero-shotBackbone=ViT-B/162025.03 | 65.3 | 65.4 | 65.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CLIPEvaluation Protocol=Zero-shot2025.03 | 65.3 | 65.4 | 65.3 | 0 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Zero-shotExtra data=–, Train Params.=–2026.04 | 65.3 | 65.4 | 65.3 | — | 65.3 | — | — | — | — | — | — | — | — | — | — | — | |
| Zero-shot2026.05 | 65.3 | 65.4 | 65.3 | 0 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Zero-shot2026.05 | 65.3 | 65.4 | 65.3 | 0 | — | — | — | — | — | — | — | — | — | — | — | — | |
| LEBAProtocol=Transfer2026.05 | 65.1 | 65.1 | — | — | — | — | 88.7 | 60.2 | 89.3 | 71.1 | 65.1 | 18.4 | 88.5 | 45.9 | 55.3 | 68.1 | |
| S-Prompts2026.05 | 65.1 | 45.3 | 83.8 | -20.1 | — | — | — | — | — | — | — | — | — | — | — | — | |
| S-PromptsArch. Kept=✗, w/o Extra Data=✓2026.06 | 65.1 | 45.3 | 83.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MoE-AdaptersProtocol=Transfer2026.05 | 64.3 | 64.3 | — | — | — | — | 88.8 | 59.5 | 89.1 | 69.9 | 64.4 | 18.1 | 86.9 | 43.7 | 54.6 | 68.2 | |
| ZSCLProtocol=Transfer2026.05 | 64.1 | 64.1 | — | — | — | — | 88.3 | 57.5 | 84.7 | 68.1 | 64.8 | 21.1 | 88.2 | 45.3 | 55.2 | 68.2 | |
| L2P2026.05 | 62.5 | 42.5 | 82.3 | -22.9 | — | — | — | — | — | — | — | — | — | — | — | — | |
| L2PArch. Kept=✗, w/o Extra Data=✓2026.06 | 62.5 | 42.5 | 82.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LwFBackbone=ViT-B/162025.03 | 62.2 | 53.2 | 71.9 | -5.3 | — | — | — | — | — | — | — | — | — | — | — | — | |
| LwF2025.03 | 62.2 | 53.2 | 71.9 | -12.2 | — | — | — | — | — | — | — | — | — | — | — | — | |
| LwFExtra data=✓, Train Params.=149.6 M2026.04 | 62.2 | 53.2 | 71.9 | — | 62.4 | — | — | — | — | — | — | — | — | — | — | — | |
| LwFProtocol=Avg.2026.05 | 62.2 | — | — | — | — | 49 | 77 | 92.1 | 85.9 | 66.5 | 67.2 | 20.9 | 84.7 | 44.6 | 45.5 | 50.5 | |
| LwF2026.05 | 62.2 | 53.2 | 71.9 | -12.2 | — | — | — | — | — | — | — | — | — | — | — | — | |
| LwFArch. Kept=✓, w/o Extra Data=✗2026.06 | 62.2 | 53.2 | 71.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| WiSE-FTBackbone=ViT-B/162025.03 | 61.5 | 51 | 72.2 | -5 | — | — | — | — | — | — | — | — | — | — | — | — | |
| WiSE-FT2025.03 | 61.5 | 51 | 72.2 | -14.4 | — | — | — | — | — | — | — | — | — | — | — | — | |
| WiSE-FTExtra data=✓, Train Params.=149.6 M2026.04 | 61.5 | 51 | 72.2 | — | 61.6 | — | — | — | — | — | — | — | — | — | — | — | |
| WiSE-FTProtocol=Avg.2026.05 | 61.5 | — | — | — | — | 52.6 | 79.3 | 91.9 | 83.9 | 63.4 | 65.2 | 23.3 | 83.7 | 45.4 | 40 | 48.2 | |
| WiSE-FT2026.05 | 61.5 | 51.1 | 72.2 | -14.3 | — | — | — | — | — | — | — | — | — | — | — | — | |
| WiSE-FTArch. Kept=✓, w/o Extra Data=✗2026.06 | 61.5 | 51.1 | 72.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LwF-VRBackbone=ViT-B/162025.03 | 60.6 | 53.1 | 68.3 | -3.9 | — | — | — | — | — | — | — | — | — | — | — | — | |
| LwF-VR2025.03 | 60.6 | 53.1 | 68.3 | -12.3 | — | — | — | — | — | — | — | — | — | — | — | — | |
| LwF-VR2026.05 | 60.6 | 53.1 | 68.2 | -12.3 | — | — | — | — | — | — | — | — | — | — | — | — | |
| LwF-VRArch. Kept=✓, w/o Extra Data=✗2026.06 | 60.6 | 53.1 | 68.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LwF-VR2026.05 | 60.1 | 54.4 | 63.5 | -11 | — | — | — | — | — | — | — | — | — | — | — | — | |
| iCaRLBackbone=ViT-B/162025.03 | 56.9 | 50.9 | 71.6 | -5.6 | — | — | — | — | — | — | — | — | — | — | — | — | |
| iCaRL2025.03 | 56.9 | 50.9 | 71.6 | -14.5 | — | — | — | — | — | — | — | — | — | — | — | — | |
| iCaRLExtra data=✓, Train Params.=149.6 M2026.04 | 56.9 | 50.9 | 71.6 | — | 59.8 | — | — | — | — | — | — | — | — | — | — | — | |
| iCaRLProtocol=Avg.2026.05 | 56.9 | — | — | — | — | 52 | 75.9 | 77.4 | 74.6 | 58.4 | 59.3 | 11.7 | 79.6 | 42.1 | 43.2 | 51.7 | |
| iCaRL2026.05 | 56.9 | 50.9 | 71.6 | -14.5 | — | — | — | — | — | — | — | — | — | — | — | — | |
| iCaRLArch. Kept=✓, w/o Extra Data=✗2026.06 | 56.9 | 50.9 | 71.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Continual FinetuneBackbone=ViT-B/162025.03 | 56.2 | 46.6 | 67.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Continual FT2025.03 | 56.2 | 46.6 | 67.4 | -18.8 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Continual-FT2026.05 | 56.2 | 46.6 | 67.4 | -18.8 | — | — | — | — | — | — | — | — | — | — | — | — | |
| WiSE-FT2026.05 | 55.1 | 53.7 | 51.8 | -11.7 | — | — | — | — | — | — | — | — | — | — | — | — | |
| LwF2026.05 | 54.7 | 48.6 | 56.8 | -16.8 | — | — | — | — | — | — | — | — | — | — | — | — | |
| LwFProtocol=Transfer2026.05 | 53.2 | 53.2 | — | — | — | — | 87.8 | 58.5 | 71.9 | 46.6 | 57.3 | 12.8 | 81.4 | 34.5 | 34.5 | 46.8 | |
| WiSE-FTProtocol=Transfer2026.05 | 51.1 | 51.1 | — | — | — | — | 87.2 | 57.6 | 67 | 45 | 54 | 12.9 | 78.6 | 35.5 | 28.4 | 44.3 | |
| iCaRLProtocol=Transfer2026.05 | 50.9 | 50.9 | — | — | — | — | 86.1 | 51.8 | 67.6 | 50.4 | 57.9 | 11 | 72.3 | 31.2 | 32.7 | 48.1 | |
| Continual-FT2026.05 | 49 | 49.2 | 42.8 | -16.2 | — | — | — | — | — | — | — | — | — | — | — | — |