Memory task on Long-range memory L=24 (test)
100AccuracyFlowYB (ours, integrable)
Evaluation results
| Method | Links | |
|---|---|---|
| FlowYB (ours, integrable)params=1,970, sequence length (L)=242026.06 | 100 | |
| Attentionparams=5,594, sequence length (L)=242026.06 | 100 | |
| Nonlinear MLP-Mixerparams=3,770, sequence length (L)=242026.06 | 100 | |
| Diagonal SSM (S4D-like)params=1,466, sequence length (L)=242026.06 | 83.3 | |
| Orthogonal RNNparams=2,570, sequence length (L)=242026.06 | 75.2 | |
| No-mixing (control)params=1,394, sequence length (L)=242026.06 | 51.2 |