Character-level language modeling on Tiny Shakespeare (val)
1.355Validation LossEGA-MORLET
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| EGA-MORLETMechanism=EGA + MOPE2026.05 | 1.355 | — | — | — | — | 0.119 | |
| EGA-MORLETEncoding=MOPE + EGA2026.05 | 1.355 | — | — | — | — | 0.119 | |
| EGA-1Mechanism=energy gate alone2026.05 | 1.3821 | — | — | — | — | 0.092 | |
| EGA-1Encoding=learned emb + EGA2026.05 | 1.3821 | — | — | — | — | 0.092 | |
| BaselineParams=4,782,336, alpha=1.0002026.04 | 1.4422 | — | — | — | 2.0807 | — | |
| TSAParams=4,864,901, alpha=0.726, TLOps saved=22.8%2026.04 | 1.4482 | — | — | — | 2.0893 | — | |
| PE-ROPEMechanism=rotary (relative)2026.05 | 1.4637 | — | — | — | — | 0.011 | |
| PE-ROPEEncoding=ROPE (relative)2026.05 | 1.4637 | — | — | — | — | 0.011 | |
| BASE-DOTMechanism=dot product + learned PE2026.05 | 1.4742 | — | — | — | — | — | |
| BASE-DOTEncoding=learned embedding2026.05 | 1.4742 | — | — | — | — | — | |
| PE-MORLETMechanism=MOPE alone2026.05 | 1.506 | — | — | — | — | 0.032 | |
| PE-MORLETEncoding=MOPE (absolute)2026.05 | 1.506 | — | — | — | — | -0.032 | |
| PE-SINCOSMechanism=fixed sin/cos2026.05 | 1.5863 | — | — | — | — | 0.112 | |
| PE-SINCOSEncoding=sin/cos (fixed)2026.05 | 1.5863 | — | — | — | — | -0.112 | |
| MPObond dimension (χ)=16, Params=191 872, Ratio=5.3×2026.03 | 1.85 | 51.6 | -1.2 | 3.72 | — | — | |
| MPObond dimension (χ)=32, Params=408 832, Ratio=2.5×2026.03 | 1.85 | 52.4 | -0.4 | 2.59 | — | — | |
| DenseParams=1 020 224, Ratio=1.0×2026.03 | 1.86 | 52.8 | — | 1.65 | — | — | |
| MPObond dimension (χ)=8, Params=110 592, Ratio=9.2×2026.03 | 1.95 | 46.8 | -6 | 4.45 | — | — | |
| MPObond dimension (χ)=4, Params=78 336, Ratio=13.0×2026.03 | 2.22 | 36.8 | -16 | 4.15 | — | — |