Synthetic Sequence Modeling on Synthetic Tasks Dataset (test)
100RecallLSTM
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| LSTMNumber of layers=1, Hidden dimension=61442026.02 | 100 | 100 | 75 | 91.6 | |
| Mamba2Number of layers=1, Hidden dimension=61442026.02 | 100 | 80.7 | 72 | 84.2 | |
| Deltanet[-1,1]Number of layers=1, Hidden dimension=61442026.02 | 100 | 89.2 | 78.2 | 89.2 | |
| Deltaproduct4[-1,1]Number of layers=1, Hidden dimension=61442026.02 | 100 | 100 | 71.7 | 90.6 | |
| BD-LRUBlock size (m)=2, Number of layers=1, Hidden dimension=61442026.02 | 100 | 96.2 | 76 | 90.8 | |
| BD-LRUBlock size (m)=3, Number of layers=1, Hidden dimension=61442026.02 | 100 | 98 | 76.2 | 91.6 | |
| BD-LRUBlock size (m)=4, Number of layers=1, Hidden dimension=61442026.02 | 100 | 98.3 | 78.5 | 92.2 | |
| BD-LRUBlock size (m)=5, Number of layers=1, Hidden dimension=61442026.02 | 100 | 98.5 | 78.2 | 92.2 | |
| BD-LRUBlock size (m)=6, Number of layers=1, Hidden dimension=61442026.02 | 100 | 98 | 77.5 | 91.8 | |
| BD-LRUBlock size (m)=8, Number of layers=1, Hidden dimension=61442026.02 | 100 | 99.2 | 74.8 | 91.3 | |
| BD-LRUBlock size (m)=16, Number of layers=1, Hidden dimension=61442026.02 | 100 | 99.8 | 72.5 | 90.7 | |
| H-LRUWindow size (m)=3, Number of layers=1, Hidden dimension=61442026.02 | 100 | 85.5 | 77.2 | 87.6 | |
| H-LRUWindow size (m)=4, Number of layers=1, Hidden dimension=61442026.02 | 100 | 84.5 | 77.5 | 87.3 | |
| H-LRUWindow size (m)=5, Number of layers=1, Hidden dimension=61442026.02 | 100 | 83.8 | 77.5 | 87.1 | |
| H-LRUWindow size (m)=6, Number of layers=1, Hidden dimension=61442026.02 | 100 | 81.8 | 77.5 | 86.4 | |
| H-LRUWindow size (m)=8, Number of layers=1, Hidden dimension=61442026.02 | 100 | 81 | 76.8 | 85.9 | |
| H-LRUWindow size (m)=16, Number of layers=1, Hidden dimension=61442026.02 | 100 | 68 | 70.5 | 79.5 | |
| H-LRUWindow size (m)=2, Number of layers=1, Hidden dimension=61442026.02 | 99.8 | 85.5 | 77 | 87.4 | |
| H-LRUWindow size (m)=1, Number of layers=1, Hidden dimension=61442026.02 | 78.5 | 84.8 | 76 | 79.7 | |
| BD-LRUBlock size (m)=1, Number of layers=1, Hidden dimension=61442026.02 | 77.5 | 83.5 | 72.5 | 77.8 |