List your company

Induction head on Synthetic sequence modeling Induction head

100AccuracyTransformer
2.302427.666253.0378.3938Dec 5, 2023

Evaluation results

MethodLinks
TransformerAttention=Softmax2023.12100
MIMOFormerN=2, variant=att., Attention=FAVOR+2023.1299.4
MIMOFormerN=2, variant=att., Attention=DPFP2023.1298.56
PerformerAttention=FAVOR+2023.1231.58
DataMUXN=2, Attention=Softmax2023.126.06