Classification on LRA ListOps N=2000 (test)
61.48AccuracyFull model
Evaluation Results
| Method | Links | |
|---|---|---|
| Full modelModel Backbone=S5, Pruning Ratio=0%, State Dimension=162024.11 | 61.48 | |
| Uniform H_∞Model Backbone=S5, Pruning Ratio=0%, State Dimension=162024.11 | 61.48 | |
| Global H_∞Model Backbone=S5, Pruning Ratio=0%, State Dimension=162024.11 | 61.48 | |
| LASTModel Backbone=S5, Pruning Ratio=0%, State Dimension=162024.11 | 61.48 | |
| Full modelModel Backbone=S4D, Pruning Ratio=0%, State Dimension=642024.11 | 56.42 | |
| LASTModel Backbone=S4D, Pruning Ratio=10%, State Dimension=582024.11 | 56.27 | |
| Uniform H_∞Model Backbone=S4D, Pruning Ratio=10%, State Dimension=582024.11 | 55.82 | |
| Global H_∞Model Backbone=S4D, Pruning Ratio=10%, State Dimension=582024.11 | 49.95 | |
| Angulargamma=82025.10 | 42.2 | |
| RACEP=2, L=22025.10 | 41.9 | |
| RACEP=4, L=42025.10 | 41.6 | |
| FlashAttention22025.10 | 41.4 | |
| RACEP=3, L=32025.10 | 41.3 | |
| YOSOL=162025.10 | 41 | |
| Performerrank=2562025.10 | 40.2 | |
| Linear2025.10 | 39.6 | |
| Paramixer (CHORD)Sequence Length=20002022.04 | 39.57 | |
| Linformerk=1282025.10 | 38.9 | |
| Transformer-LSReference Source=[25], Sequence Length=20002022.04 | 38.36 | |
| Paramixer (CDIL)Sequence Length=20002022.04 | 37.78 | |
| LinformerReference Source=[24], Sequence Length=20002022.04 | 37.7 | |
| LinformerReference Source=[25], Sequence Length=20002022.04 | 37.38 | |
| NyströmformerReference Source=[25], Sequence Length=20002022.04 | 37.34 | |
| ReformerReference Source=[24], Sequence Length=20002022.04 | 37.27 | |
| LinformerReference Source=[15], Sequence Length=20002022.04 | 37.25 | |
| NyströmformerReference Source=[15], Sequence Length=20002022.04 | 37.15 | |
| TransformerReference Source=[25], Sequence Length=20002022.04 | 37.13 | |
| TransformerReference Source=[15], Sequence Length=20002022.04 | 37.1 | |
| RFA-GaussianReference Source=[18], Sequence Length=20002022.04 | 36.8 | |
| ReformerReference Source=[25], Sequence Length=20002022.04 | 36.44 | |
| TransformerReference Source=[24], Sequence Length=20002022.04 | 36.37 | |
| BigBirdReference Source=[24], Sequence Length=20002022.04 | 36.06 | |
| LongformerReference Source=[24], Sequence Length=20002022.04 | 35.63 | |
| PerformerReference Source=[25], Sequence Length=20002022.04 | 32.78 | |
| ReformerReference Source=[15], Sequence Length=20002022.04 | 19.05 | |
| PerformerReference Source=[15], Sequence Length=20002022.04 | 18.8 | |
| PerformerReference Source=[24], Sequence Length=20002022.04 | 18.01 | |
| Sparse TransformerReference Source=[24], Sequence Length=20002022.04 | 17.07 | |
| Linear TransformerReference Source=[24], Sequence Length=20002022.04 | 16.13 |