Continual Learning on CIFAR100 Split
95.38Average Per-Task AccuracyMulti-Head (multi-task)
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Multi-Head (multi-task)2021.06 | 95.38 | — | — | — | — | — | — | — | — | — | |
| Model ZooReplay=true, Single Epoch=false2021.06 | 94.99 | — | — | — | — | — | — | — | — | — | |
| NoRGaBackbone=ViT-B/16, Pre-training=Sup-21K2024.05 | 94.48 | — | — | — | — | — | — | — | — | — | |
| Model Zoo-ResnetReplay=true, Single Epoch=false2021.06 | 93.15 | — | — | — | — | — | — | — | — | — | |
| RanPACBackbone=ViT-B/16, Pre-training=Sup-21K2024.05 | 92.2 | — | — | — | — | — | — | — | — | — | |
| Model Zoo-smallReplay=true, Single Epoch=false2021.06 | 92.06 | — | — | — | — | — | — | — | — | — | |
| IsolatedReplay=false, Single Epoch=false2021.06 | 91.9 | — | — | — | — | — | — | — | — | — | |
| Isolated-smallReplay=false, Single Epoch=false2021.06 | 90.18 | — | — | — | — | — | — | — | — | — | |
| Model Zoo-small (10% replay)Replay=true, Single Epoch=false2021.06 | 89.76 | — | — | — | — | — | — | — | — | — | |
| Isolated-ResnetReplay=false, Single Epoch=false2021.06 | 88.95 | — | — | — | — | — | — | — | — | — | |
| ADAM + AdapterBackbone=ViT-B/16, Pre-training=Sup-21K2024.05 | 87.29 | — | — | — | — | — | — | — | — | — | |
| ADAM + SSFBackbone=ViT-B/16, Pre-training=Sup-21K2024.05 | 85.27 | — | — | — | — | — | — | — | — | — | |
| ADAM + VPT-DBackbone=ViT-B/16, Pre-training=Sup-21K2024.05 | 85.04 | — | — | — | — | — | — | — | — | — | |
| ER-ReservoirReplay=true, Single Epoch=true2021.06 | 68.5 | — | — | — | — | — | — | — | — | — | |
| GEMReplay=true, Single Epoch=true2021.06 | 67.8 | — | — | — | — | — | — | — | — | — | |
| MEGA-IIReplay=true2021.06 | 66.12 | — | — | — | — | — | — | — | — | — | |
| MC-SGDReplay=true, Single Epoch=true2021.06 | 63.3 | — | — | — | — | — | — | — | — | — | |
| TAGReplay=false, Single Epoch=true2021.06 | 62.79 | — | — | — | — | — | — | — | — | — | |
| A-GEMReplay=true, Single Epoch=true2021.06 | 62.3 | — | — | — | — | — | — | — | — | — | |
| Stable-SGDReplay=false, Single Epoch=true2021.06 | 59.9 | — | — | — | — | — | — | — | — | — | |
| Prog-NetsReplay=false, Single Epoch=false2021.06 | 59.2 | — | — | — | — | — | — | — | — | — | |
| Refresh (IM)Buffer=20002025.12 | 55.1 | — | — | — | — | — | — | — | — | — | |
| DER++ (IM)Buffer=20002025.12 | 54.3 | — | — | — | — | — | — | — | — | — | |
| DER (IM)Buffer=20002025.12 | 53.6 | — | — | — | — | — | — | — | — | — | |
| RefreshBuffer=20002025.12 | 53.6 | — | — | — | — | — | — | — | — | — | |
| DER++ (EM)Buffer=20002025.12 | 52.6 | — | — | — | — | — | — | — | — | — | |
| DER++Buffer=20002025.12 | 52.5 | — | — | — | — | — | — | — | — | — | |
| DERBuffer=20002025.12 | 51.9 | — | — | — | — | — | — | — | — | — | |
| DER (SI)Buffer=20002025.12 | 50.4 | — | — | — | — | — | — | — | — | — | |
| DER++ (SI)Buffer=20002025.12 | 50.3 | — | — | — | — | — | — | — | — | — | |
| Refresh (IM)Buffer=10002025.12 | 49.5 | — | — | — | — | — | — | — | — | — | |
| DER (EM)Buffer=20002025.12 | 49.3 | — | — | — | — | — | — | — | — | — | |
| STAR (IM)Buffer=20002025.12 | 48.8 | — | — | — | — | — | — | — | — | — | |
| DER (IM)Buffer=10002025.12 | 48.7 | — | — | — | — | — | — | — | — | — | |
| RefreshBuffer=10002025.12 | 48 | — | — | — | — | — | — | — | — | — | |
| DER++ (IM)Buffer=10002025.12 | 47.3 | — | — | — | — | — | — | — | — | — | |
| Adaptive TrainingTraining Mode=Adaptive2026.02 | 47.19 | — | — | — | — | — | — | — | 24.15 | — | |
| Adaptive TrainingTraining Mode=Adaptive, Framework=Adaptive Hybrid Update Framework2026.02 | 47.19 | — | — | — | — | — | — | — | 24.15 | — | |
| STARBuffer=20002025.12 | 46.6 | — | — | — | — | — | — | — | — | — | |
| ER (IM)Buffer=20002025.12 | 46.2 | — | — | — | — | — | — | — | — | — | |
| DER++Buffer=10002025.12 | 45.9 | — | — | — | — | — | — | — | — | — | |
| DER++ (EM)Buffer=10002025.12 | 44.6 | — | — | — | — | — | — | — | — | — | |
| STAR (IM)Buffer=10002025.12 | 43.4 | — | — | — | — | — | — | — | — | — | |
| Frozen TrainingTraining Mode=Frozen2026.02 | 43.38 | — | — | — | — | — | — | — | 23.96 | — | |
| Frozen TrainingTraining Mode=Frozen2026.02 | 43.38 | — | — | — | — | — | — | — | 23.96 | — | |
| Refresh (IM)Buffer=5002025.12 | 42.7 | — | — | — | — | — | — | — | — | — | |
| DERBuffer=10002025.12 | 42.6 | — | — | — | — | — | — | — | — | — | |
| DER (EM)Buffer=10002025.12 | 42.6 | — | — | — | — | — | — | — | — | — | |
| EWCReplay=false, Single Epoch=false2021.06 | 42.4 | — | — | — | — | — | — | — | — | — | |
| DER (SI)Buffer=10002025.12 | 42.3 | — | — | — | — | — | — | — | — | — | |
| DER++ (SI)Buffer=10002025.12 | 42.3 | — | — | — | — | — | — | — | — | — | |
| Initialized TrainingTraining Mode=Initialized2026.02 | 42.27 | — | — | — | — | — | — | — | 24.84 | — | |
| Initialized TrainingTraining Mode=Initialized2026.02 | 42.27 | — | — | — | — | — | — | — | 24.84 | — | |
| DER++ (IM)Buffer=5002025.12 | 41.6 | — | — | — | — | — | — | — | — | — | |
| RefreshBuffer=5002025.12 | 40.8 | — | — | — | — | — | — | — | — | — | |
| DER (IM)Buffer=5002025.12 | 40.6 | — | — | — | — | — | — | — | — | — | |
| DER++ (EM)Buffer=5002025.12 | 40.5 | — | — | — | — | — | — | — | — | — | |
| DER++Buffer=5002025.12 | 40.4 | — | — | — | — | — | — | — | — | — | |
| ER (IM)Buffer=10002025.12 | 39.8 | — | — | — | — | — | — | — | — | — | |
| ER (EM)Buffer=20002025.12 | 38.3 | — | — | — | — | — | — | — | — | — | |
| ER (EWC)Buffer=20002025.12 | 37.4 | — | — | — | — | — | — | — | — | — | |
| STARBuffer=10002025.12 | 36.9 | — | — | — | — | — | — | — | — | — | |
| ER (SI)Buffer=20002025.12 | 36.4 | — | — | — | — | — | — | — | — | — | |
| ERBuffer=20002025.12 | 36.3 | — | — | — | — | — | — | — | — | — | |
| STAR (IM)Buffer=5002025.12 | 34.7 | — | — | — | — | — | — | — | — | — | |
| DER++ (SI)Buffer=5002025.12 | 33.7 | — | — | — | — | — | — | — | — | — | |
| ER (IM)Buffer=5002025.12 | 33.6 | — | — | — | — | — | — | — | — | — | |
| DERBuffer=5002025.12 | 33.6 | — | — | — | — | — | — | — | — | — | |
| DER (EM)Buffer=5002025.12 | 32.3 | — | — | — | — | — | — | — | — | — | |
| ER (EWC)Buffer=10002025.12 | 31.4 | — | — | — | — | — | — | — | — | — | |
| ERBuffer=10002025.12 | 29.6 | — | — | — | — | — | — | — | — | — | |
| DER (SI)Buffer=5002025.12 | 29.5 | — | — | — | — | — | — | — | — | — | |
| ER (SI)Buffer=10002025.12 | 29 | — | — | — | — | — | — | — | — | — | |
| ER (EM)Buffer=10002025.12 | 28.8 | — | — | — | — | — | — | — | — | — | |
| STARBuffer=5002025.12 | 28.6 | — | — | — | — | — | — | — | — | — | |
| DER (EWC)Buffer=20002025.12 | 25 | — | — | — | — | — | — | — | — | — | |
| Grow+Rand. Smooth-LeakyCompactness=20%2026.05 | 25 | — | — | — | — | — | — | — | — | 57.7 | |
| Grow+Rand. Smooth-LeakyCompactness=50%2026.05 | 24.7 | — | — | — | — | — | — | — | — | 46.9 | |
| Grow+Rand. Smooth-LeakyCompactness=30%2026.05 | 24.5 | — | — | — | — | — | — | — | — | 57.8 | |
| ER (EWC)Buffer=5002025.12 | 24.2 | — | — | — | — | — | — | — | — | — | |
| Grow+Rand. Smooth-LeakyCompactness=40%2026.05 | 24.2 | — | — | — | — | — | — | — | — | 45.9 | |
| DER++ (EWC)Buffer=20002025.12 | 22.8 | — | — | — | — | — | — | — | — | — | |
| Grow+Moment TransplantCompactness=30%2026.05 | 22.1 | — | — | — | — | — | — | — | — | 55.6 | |
| Grow+Moment TransplantCompactness=50%2026.05 | 22.1 | — | — | — | — | — | — | — | — | 53.5 | |
| PruneCompactness=40%2026.05 | 21.9 | — | — | — | — | — | — | — | — | 55.1 | |
| GrowCompactness=40%2026.05 | 21.9 | — | — | — | — | — | — | — | — | 51.9 | |
| Grow+GradMaxCompactness=40%2026.05 | 21.9 | — | — | — | — | — | — | — | — | 52.3 | |
| PruneCompactness=50%2026.05 | 21.9 | — | — | — | — | — | — | — | — | 57.8 | |
| ER (SI)Buffer=5002025.12 | 21.8 | — | — | — | — | — | — | — | — | — | |
| Grow+GradMaxCompactness=50%2026.05 | 21.8 | — | — | — | — | — | — | — | — | 51 | |
| ERBuffer=5002025.12 | 21.6 | — | — | — | — | — | — | — | — | — | |
| Grow+GradMaxCompactness=30%2026.05 | 21.6 | — | — | — | — | — | — | — | — | 53.2 | |
| Grow+Moment TransplantCompactness=40%2026.05 | 21.5 | — | — | — | — | — | — | — | — | 53 | |
| GrowCompactness=50%2026.05 | 21.5 | — | — | — | — | — | — | — | — | 52 | |
| PruneCompactness=20%2026.05 | 21.3 | — | — | — | — | — | — | — | — | 55.1 | |
| PruneCompactness=30%2026.05 | 21.3 | — | — | — | — | — | — | — | — | 54.5 | |
| Grow+TwoSpeedCompactness=30%2026.05 | 21.2 | — | — | — | — | — | — | — | — | 50.5 | |
| Grow+TwoSpeedCompactness=40%2026.05 | 21.2 | — | — | — | — | — | — | — | — | 51.1 | |
| Grow+TwoSpeedCompactness=50%2026.05 | 21.2 | — | — | — | — | — | — | — | — | 51.3 | |
| GrowCompactness=30%2026.05 | 21.1 | — | — | — | — | — | — | — | — | 49.4 |