Token Retrieval on Induction Heads (val)
100AccuracyYB-Mixer
Evaluation results
| Method | Links | |
|---|---|---|
| YB-MixerSequence Length (L)=256, Parameters (M)=2.5, Model Dimension=256, Depth=8, Epochs=50, Seed=single2026.06 | 100 | |
| LRUSequence Length (L)=256, Parameters (M)=2.4, Model Dimension=256, Depth=8, Epochs=50, Seed=single2026.06 | 100 | |
| FNetSequence Length (L)=256, Parameters (M)=2.4, Model Dimension=256, Depth=8, Epochs=50, Seed=single2026.06 | 9.4 | |
| S4D-LinSequence Length (L)=256, Parameters (M)=3.2, Model Dimension=256, Depth=8, Epochs=50, Seed=single2026.06 | 8.7 | |
| TransformerSequence Length (L)=256, Parameters (M)=4.4, Model Dimension=256, Depth=8, Epochs=50, Seed=single2026.06 | 8.7 |