Continual Supervised Learning on CIFAR Random Label
98.42Total Average Online Task AccuracyRand. Smooth-Leaky
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Rand. Smooth-LeakyActivation=Rand. Smooth-Leaky2025.09 | 98.42 | — | |
| Smooth-LeakyActivation=Smooth-Leaky2025.09 | 98.36 | — | |
| Leaky-ReLUActivation=Leaky-ReLU2025.09 | 98.34 | — | |
| RReLUActivation=RReLU2025.09 | 98.02 | — | |
| PruneCompactness=50%2026.05 | 97.9 | 19.6 | |
| PruneCompactness=40%2026.05 | 97.8 | 19.6 | |
| PruneCompactness=20%2026.05 | 97.6 | 19.6 | |
| PruneCompactness=30%2026.05 | 97.6 | 19.6 | |
| Grow+Rand. Smooth-LeakyCompactness=50%2026.05 | 97.1 | 19.6 | |
| PReLUActivation=PReLU2025.09 | 96.86 | — | |
| Grow+Rand. Smooth-LeakyCompactness=40%2026.05 | 96.8 | 19.6 | |
| Deep FourierActivation=Deep Fourier2025.09 | 96.24 | — | |
| Grow+Rand. Smooth-LeakyCompactness=30%2026.05 | 96.1 | 19.6 | |
| RationalActivation=Rational2025.09 | 94.82 | — | |
| Grow+Rand. Smooth-LeakyCompactness=20%2026.05 | 94.4 | 19.6 | |
| CReLUActivation=CReLU2025.09 | 92.9 | — | |
| Swish (SiLU)Activation=Swish (SiLU)2025.09 | 87.4 | — | |
| SeLUActivation=SeLU2025.09 | 84.61 | — | |
| SwiGLUActivation=SwiGLU2025.09 | 83.06 | — | |
| TanhActivation=Tanh2025.09 | 58.56 | — | |
| eLUActivation=eLU2025.09 | 57.45 | — | |
| SigmoidActivation=Sigmoid2025.09 | 52.24 | — | |
| DenseCompactness=20%2026.05 | 46.3 | 17.3 | |
| DenseCompactness=30%2026.05 | 46.3 | 17.3 | |
| DenseCompactness=40%2026.05 | 46.3 | 17.3 | |
| DenseCompactness=50%2026.05 | 46.3 | 17.3 | |
| GeLUActivation=GeLU2025.09 | 42.85 | — | |
| CeLUActivation=CeLU2025.09 | 29.64 | — | |
| ReLUActivation=ReLU2025.09 | 25.79 | — | |
| Grow+Moment TransplantCompactness=30%2026.05 | 19.5 | 13.9 | |
| Grow+Net2WiderCompactness=40%2026.05 | 18 | 13.4 | |
| Grow+Moment TransplantCompactness=40%2026.05 | 17.4 | 13.6 | |
| Grow+Moment TransplantCompactness=50%2026.05 | 17.3 | 13.6 | |
| Grow+TwoSpeedCompactness=50%2026.05 | 15.8 | 13 | |
| Grow+TwoSpeedCompactness=30%2026.05 | 15.4 | 12.9 | |
| GrowCompactness=50%2026.05 | 15.4 | 13 | |
| Grow+Moment TransplantCompactness=20%2026.05 | 15.2 | 13.4 | |
| Grow+Net2WiderCompactness=30%2026.05 | 15.1 | 12.9 | |
| GrowCompactness=30%2026.05 | 14.9 | 12.8 | |
| Grow+Net2WiderCompactness=50%2026.05 | 14.8 | 12.7 | |
| Grow+TwoSpeedCompactness=40%2026.05 | 14.6 | 12.9 | |
| GrowCompactness=40%2026.05 | 14.1 | 12.8 | |
| Grow+GradMaxCompactness=30%2026.05 | 13.9 | 12.7 | |
| Grow+TwoSpeedCompactness=20%2026.05 | 13.8 | 12.8 | |
| Grow+GradMaxCompactness=50%2026.05 | 13.8 | 12.7 | |
| Grow+GradMaxCompactness=20%2026.05 | 13.7 | 12.6 | |
| Grow+GradMaxCompactness=40%2026.05 | 13.6 | 12.5 | |
| GrowCompactness=20%2026.05 | 13.5 | 12.8 | |
| Grow+Net2WiderCompactness=20%2026.05 | 13.1 | 12.5 |