Scaling-law extrapolation on TinyStories high-C holdout
0.095RMSE (log space)Muennighoff
Evaluation results
| Method | Links | |
|---|---|---|
| Muennighoff2026.05 | 0.095 | |
| Farseer2026.05 | 0.119 | |
| Chinchilla2026.05 | 0.173 | |
| Ours (Eq. (2))2026.05 | 0.184 | |
| BNSL k=22026.05 | 0.324 | |
| M4 (D-axis)2026.05 | 0.337 |