List your company
ResearchFormerly WizWand

Sequence Classification on ListOps

98.85Accuracy (%)Transformer (Full Attn)
12.7935.132557.47579.8175Sep 30, 2021Jul 12, 2022Apr 23, 2023Feb 2, 2024Nov 13, 2024Aug 25, 2025Jun 6, 2026
Updated 4mo ago

Evaluation results

MethodLinks
Transformer (Full Attn)Source=This work, Tokenization=GPT-2 BPE2026.0698.85
CHIAR DCT+AttnSource=This work, Tokenization=GPT-2 BPE2026.0663.35
CMRUDepth (r)=6, State dimension (d)=322026.0561.39
αCMRUDepth (r)=6, State dimension (d)=322026.0561.36
αCMRUDepth (r)=3, State dimension (d)=322026.0560.86
CMRUDepth (r)=3, State dimension (d)=322026.0560.42
MINGRUDepth (r)=6, State dimension (d)=322026.0559.69
HKT-Larged=512, Overhead=1.313×2026.0455.6
HKT-Mediumd=256, Overhead=1.313×2026.0455.5
HKT-Smalld=128, Overhead=1.313×2026.0455.1
MINGRUDepth (r)=3, State dimension (d)=322026.0555.07
MHA (retrained)Overhead=1.00×2026.0450.33
TransEvolve-randomFF-1hidden size=256, configuration=randomFF-12021.0943.2
TransEvolve-fullFF-1hidden size=256, configuration=fullFF-12021.0942.2
LRUDepth (r)=3, State dimension (d)=322026.0540.45
MINGRUDepth (r)=1, State dimension (d)=322026.0539.4
LRUDepth (r)=6, State dimension (d)=322026.0539.37
TransEvolve-randomFF-2hidden size=256, configuration=randomFF-22021.0939.1
CMRUDepth (r)=1, State dimension (d)=322026.0538.84
αCMRUDepth (r)=1, State dimension (d)=322026.0538.74
LRUDepth (r)=1, State dimension (d)=322026.0538.64
TransEvolve-fullFF-2hidden size=256, configuration=fullFF-22021.0937.8
Reformer2021.0937.3
Synthesizer2021.0937
Random Feature Attention (RFA)2021.0936.8
Transformerhidden size=5122021.0936.4
TransformerSource=Tay et al. 2021, Tokenization=Character-level2026.0636.37
Big Bird2021.0936
Linformer2021.0935.7
LinformerSource=Tay et al. 2021, Tokenization=Character-level2026.0635.7
FNetSource=Tay et al. 2021, Tokenization=Character-level2026.0635.33
PerformerSource=Tay et al. 2021, Tokenization=Character-level2026.0618.01
Performers2021.0918
Sinkhorn2021.0917.1
Linear attention2021.0916.1