Modular Addition on Modular Addition f_train=0.5 P=97
2,460Grokking Onset (epochs)Norm Penalty
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Norm PenaltyArchitecture=MLP, Regularization=Activation Norm Penalty2026.06 | 2,460 | 6.3 | |
| Norm PenaltyArchitecture=Transformer, Regularization=Activation Norm Penalty2026.06 | 5,200 | 1.5 | |
| Strong WD BaselineArchitecture=Transformer, Regularization=Weight Decay (10^-1)2026.06 | 8,000 | — | |
| Strong WD BaselineArchitecture=MLP, Regularization=Weight Decay (10^-1)2026.06 | 15,540 | — |