Continual Supervised Learning on Continual ImageNet
93.8Total Average Online Task AccuracyPrune
Evaluation results
| Method | Links | ||
|---|---|---|---|
| PruneCompactness=50%2026.05 | 93.8 | 50.7 | |
| PruneCompactness=40%2026.05 | 93.1 | 50.8 | |
| PruneCompactness=30%2026.05 | 92 | 50.7 | |
| PruneCompactness=20%2026.05 | 89.9 | 50.7 | |
| Rand. Smooth-LeakyActivation=Rand. Smooth-Leaky2025.09 | 86.23 | — | |
| Smooth-LeakyActivation=Smooth-Leaky2025.09 | 85.38 | — | |
| Leaky-ReLUActivation=Leaky-ReLU2025.09 | 85.28 | — | |
| RReLUActivation=RReLU2025.09 | 84.97 | — | |
| CReLUActivation=CReLU2025.09 | 84.85 | — | |
| Grow+Rand. Smooth-LeakyCompactness=50%2026.05 | 84.2 | 50.7 | |
| Grow+Rand. Smooth-LeakyCompactness=40%2026.05 | 83.5 | 50.7 | |
| Grow+Rand. Smooth-LeakyCompactness=30%2026.05 | 82.8 | 50.8 | |
| Swish (SiLU)Activation=Swish (SiLU)2025.09 | 82.64 | — | |
| PReLUActivation=PReLU2025.09 | 82.37 | — | |
| Grow+Rand. Smooth-LeakyCompactness=20%2026.05 | 81.5 | 50.9 | |
| CeLUActivation=CeLU2025.09 | 81.15 | — | |
| SeLUActivation=SeLU2025.09 | 80.98 | — | |
| RationalActivation=Rational2025.09 | 80.65 | — | |
| eLUActivation=eLU2025.09 | 80.1 | — | |
| Deep FourierActivation=Deep Fourier2025.09 | 76.03 | — | |
| GeLUActivation=GeLU2025.09 | 75.49 | — | |
| ReLUActivation=ReLU2025.09 | 73.71 | — | |
| DenseCompactness=20%2026.05 | 73.3 | 68.2 | |
| DenseCompactness=30%2026.05 | 73.3 | 68.2 | |
| DenseCompactness=40%2026.05 | 73.3 | 68.2 | |
| DenseCompactness=50%2026.05 | 73.3 | 68.2 | |
| Grow+Moment TransplantCompactness=30%2026.05 | 72.2 | 50.7 | |
| GrowCompactness=30%2026.05 | 71.9 | 50.7 | |
| Grow+GradMaxCompactness=20%2026.05 | 71.6 | 50.7 | |
| Grow+GradMaxCompactness=50%2026.05 | 71.5 | 50.6 | |
| GrowCompactness=50%2026.05 | 71.3 | 50.7 | |
| Grow+TwoSpeedCompactness=50%2026.05 | 71.2 | 50.6 | |
| TanhActivation=Tanh2025.09 | 70.97 | — | |
| Grow+Moment TransplantCompactness=50%2026.05 | 70.8 | 50.8 | |
| GrowCompactness=20%2026.05 | 70.7 | 50.7 | |
| Grow+Net2WiderCompactness=20%2026.05 | 70.7 | 50.8 | |
| Grow+TwoSpeedCompactness=40%2026.05 | 70.6 | 50.7 | |
| Grow+GradMaxCompactness=40%2026.05 | 70.5 | 50.6 | |
| GrowCompactness=40%2026.05 | 70.4 | 50.7 | |
| Grow+Moment TransplantCompactness=40%2026.05 | 70.2 | 50.7 | |
| Grow+GradMaxCompactness=30%2026.05 | 69.9 | 50.8 | |
| Grow+Net2WiderCompactness=50%2026.05 | 69.6 | 50.7 | |
| Grow+TwoSpeedCompactness=30%2026.05 | 69.3 | 50.7 | |
| Grow+Moment TransplantCompactness=20%2026.05 | 68.9 | 50.8 | |
| Grow+Net2WiderCompactness=40%2026.05 | 68.5 | 50.7 | |
| Grow+Net2WiderCompactness=30%2026.05 | 67.9 | 50.7 | |
| Grow+TwoSpeedCompactness=20%2026.05 | 67.3 | 50.7 | |
| SigmoidActivation=Sigmoid2025.09 | 63.89 | — | |
| SwiGLUActivation=SwiGLU2025.09 | 63.57 | — |