Causal Language Modeling on TinyStories (val)
8.54PPLSoftmax
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Softmax2026.06 | 8.54 | — | |
| Oscillator attentiondosc=322026.06 | 9.11 | 0.57 | |
| Oscillator attentiondosc=162026.06 | 9.35 | 0.81 | |
| Oscillator attentiondosc=82026.06 | 9.78 | 1.24 | |
| Oscillator attentiondosc=42026.06 | 10.28 | 1.74 | |
| Oscillator attentiondosc=22026.06 | 10.93 | 2.39 |