Continual Learning on Permuted MNIST
87.2Average AccuracyPrune
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| PruneCompactness=50%2026.05 | 87.2 | — | — | — | 11.7 | |
| PruneCompactness=40%2026.05 | 86.9 | — | — | — | 11.8 | |
| PruneCompactness=30%2026.05 | 86.5 | — | — | — | 11.7 | |
| PruneCompactness=20%2026.05 | 85.9 | — | — | — | 12.1 | |
| Grow+Rand. Smooth-LeakyCompactness=50%2026.05 | 81.3 | — | — | — | 15.2 | |
| DenseCompactness=20%2026.05 | 80.6 | — | — | — | 90.9 | |
| DenseCompactness=30%2026.05 | 80.6 | — | — | — | 90.9 | |
| DenseCompactness=40%2026.05 | 80.6 | — | — | — | 90.9 | |
| DenseCompactness=50%2026.05 | 80.6 | — | — | — | 90.9 | |
| Grow+Rand. Smooth-LeakyCompactness=40%2026.05 | 80 | — | — | — | 15.4 | |
| Grow+Rand. Smooth-LeakyCompactness=30%2026.05 | 78.1 | — | — | — | 15.6 | |
| Grow+Moment TransplantCompactness=50%2026.05 | 76.1 | — | — | — | 18.4 | |
| GrowCompactness=50%2026.05 | 75.7 | — | — | — | 18.3 | |
| Grow+TwoSpeedCompactness=50%2026.05 | 75.6 | — | — | — | 18.4 | |
| Grow+GradMaxCompactness=50%2026.05 | 75.5 | — | — | — | 18.3 | |
| Grow+Net2WiderCompactness=50%2026.05 | 75.4 | — | — | — | 18.4 | |
| Grow+Rand. Smooth-LeakyCompactness=20%2026.05 | 75.1 | — | — | — | 16.2 | |
| Grow+TwoSpeedCompactness=40%2026.05 | 75 | — | — | — | 18.2 | |
| Grow+Moment TransplantCompactness=40%2026.05 | 75 | — | — | — | 18.3 | |
| GrowCompactness=40%2026.05 | 74.7 | — | — | — | 18.2 | |
| Grow+GradMaxCompactness=40%2026.05 | 74.6 | — | — | — | 18.3 | |
| Grow+Net2WiderCompactness=40%2026.05 | 74.5 | — | — | — | 18.1 | |
| Grow+Moment TransplantCompactness=30%2026.05 | 74 | — | — | — | 18.2 | |
| Grow+GradMaxCompactness=30%2026.05 | 73.4 | — | — | — | 18.1 | |
| GrowCompactness=30%2026.05 | 73.3 | — | — | — | 18.2 | |
| Grow+TwoSpeedCompactness=30%2026.05 | 73.3 | — | — | — | 18.2 | |
| Grow+Net2WiderCompactness=30%2026.05 | 73.1 | — | — | — | 18.1 | |
| Grow+Moment TransplantCompactness=20%2026.05 | 72 | — | — | — | 18.1 | |
| GrowCompactness=20%2026.05 | 71.5 | — | — | — | 18.1 | |
| Grow+TwoSpeedCompactness=20%2026.05 | 71.5 | — | — | — | 18.3 | |
| Grow+GradMaxCompactness=20%2026.05 | 71.5 | — | — | — | 18.2 | |
| Grow+Net2WiderCompactness=20%2026.05 | 71.3 | — | — | — | 18.1 | |
| ALASSONumber of tasks=302019.08 | — | 94.4 | 2.7 | 1.5 | — | |
| CeLUActivation=CeLU2025.09 | — | 82.93 | — | — | — | |
| CReLUActivation=CReLU2025.09 | — | 82.66 | — | — | — | |
| Deep FourierActivation=Deep Fourier2025.09 | — | 83.69 | — | — | — | |
| eLUActivation=eLU2025.09 | — | 80.5 | — | — | — | |
| EWCNumber of tasks=302019.08 | — | 73.8 | 24.3 | 1 | — | |
| GeLUActivation=GeLU2025.09 | — | 78.97 | — | — | — | |
| Leaky-ReLUActivation=Leaky-ReLU2025.09 | — | 84.14 | — | — | — | |
| PReLUActivation=PReLU2025.09 | — | 82.62 | — | — | — | |
| Rand. Smooth-LeakyActivation=Rand. Smooth-Leaky2025.09 | — | 84.26 | — | — | — | |
| RationalActivation=Rational2025.09 | — | 80.08 | — | — | — | |
| ReLUActivation=ReLU2025.09 | — | 78.85 | — | — | — | |
| RReLUActivation=RReLU2025.09 | — | 83.95 | — | — | — | |
| SeLUActivation=SeLU2025.09 | — | 80.43 | — | — | — | |
| SINumber of tasks=302019.08 | — | 75.2 | 23.2 | 0.8 | — | |
| SigmoidActivation=Sigmoid2025.09 | — | 76.96 | — | — | — | |
| Smooth-LeakyActivation=Smooth-Leaky2025.09 | — | 84.03 | — | — | — | |
| SwiGLUActivation=SwiGLU2025.09 | — | 77.69 | — | — | — | |
| Swish (SiLU)Activation=Swish (SiLU)2025.09 | — | 83.41 | — | — | — | |
| TanhActivation=Tanh2025.09 | — | 70.32 | — | — | — |