Grokking Onset and Speedup on 3-digit Addition (NanoGPT Implementation)
9,800Grokking OnsetNorm Penalty
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Norm PenaltyArchitecture=NanoGPT, Regularization=Activation Norm Penalty2026.06 | 9,800 | 2.3 | |
| No-penalty BaselineArchitecture=NanoGPT, Regularization=None2026.06 | 22,500 | — |