Sequential Image Classification on pMNIST (test)
98.8Accuracy (Test)Focus
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Focushidden dimension=512, layers=62023.05 | 98.8 | — | |
| FlexTCN-4Parameters=241K2021.10 | 98.72 | — | |
| S4hidden dimension=512, layers=62023.05 | 98.7 | — | |
| FlexTCN-6Parameters=375K2021.10 | 98.63 | — | |
| FlexTCN MAGNet-6Parameters=375K2021.10 | 98.63 | — | |
| FlexTCN-2Parameters=108K2021.10 | 98.61 | — | |
| CKTCN MAGNET-2Parameters=105K2021.10 | 98.57 | — | |
| CKCNN-BigNumber of Parameters=1M2021.02 | 98.54 | — | |
| CKCNN-2-BigParameters=1M2021.10 | 98.54 | — | |
| pLMUParameters=165K2021.10 | 98.45 | — | |
| CKTCN FOURIER-2Parameters=105K2021.10 | 98.4 | — | |
| CKTCN GABOR-2Parameters=106K2021.10 | 98.38 | — | |
| FlexTCN Gabor-6Parameters=373K2021.10 | 98.37 | — | |
| HiPPO-LegS2020.08 | 98.3 | — | |
| HiPPOParameters=0.5M2021.10 | 98.3 | — | |
| TrellisNet2020.08 | 98.13 | — | |
| TrellisNetNumber of Parameters=8M2021.02 | 98.13 | — | |
| TrellisNetParameters=8M2021.10 | 98.13 | — | |
| CKCNNNumber of Parameters=98K2021.02 | 98 | — | |
| CKCNN-2Parameters=98K2021.10 | 98 | — | |
| Focus-H (ablation)hidden dimension=512, layers=62023.05 | 98 | — | |
| FlexTCN Fourier-6Parameters=370K2021.10 | 97.97 | — | |
| Transformer2020.08 | 97.9 | — | |
| Self-Att.Number of Parameters=0.5M2021.02 | 97.9 | — | |
| Self-Att.Parameters=0.5M2021.10 | 97.9 | — | |
| Transformerhidden dimension=512, layers=62023.05 | 97.9 | — | |
| coRNN# units=256, # params=134k2020.10 | 97.3 | — | |
| Lipschitz RNNParameters=158K2021.10 | 97.3 | — | |
| coRNNParameters=134K2021.10 | 97.3 | — | |
| TCN2020.08 | 97.2 | — | |
| dense-IndRNN2019.10 | 97.2 | — | |
| TCNNumber of Parameters=70K2021.02 | 97.2 | — | |
| TCNParameters=70K2021.10 | 97.2 | — | |
| LMU2020.08 | 97.15 | — | |
| res-IndRNNNumber of layers=122019.10 | 97.02 | — | |
| URLSTM2020.08 | 96.96 | — | |
| URLSTM2021.10 | 96.96 | — | |
| IndRNNNumber of layers=122019.10 | 96.84 | — | |
| DTRIV∞# units=512, # params=137k2020.10 | 96.8 | — | |
| coRNN# units=128, # params=34k2020.10 | 96.6 | — | |
| URGRU + Zoneout2021.10 | 96.51 | — | |
| Dilated RNN2020.08 | 96.1 | — | |
| DilRNNNumber of Parameters=44K2021.02 | 96.1 | — | |
| DilRNNParameters=44K2021.10 | 96.1 | — | |
| IndRNN2020.08 | 96 | — | |
| IndRNNNumber of layers=62019.10 | 96 | — | |
| IndRNNNumber of Parameters=83K2021.02 | 96 | — | |
| IndRNNParameters=83K2021.10 | 96 | — | |
| LSTM+Recurrent batchnorm+Zoneout2019.10 | 95.9 | — | |
| anti.sym. RNN# units=128, # params=10k2020.10 | 95.8 | — | |
| Recurrent Batch Normalization2016.06 | 95.6 | — | |
| LSTM+Recurrent batchnorm2019.10 | 95.4 | — | |
| FlexTCN SIREN-6Parameters=343K2021.10 | 95.36 | — | |
| c-LSTM2020.08 | 95.2 | — | |
| r-LSTMNumber of Parameters=0.5M2021.02 | 95.2 | — | |
| r-LSTMParameters=0.5M2021.10 | 95.2 | — | |
| LSTM2020.08 | 95.11 | — | |
| FastGRNN# units=128, # params=18k2020.10 | 94.8 | — | |
| GRU# units=256, # params=200k2020.10 | 94.1 | — | |
| Soft D-NTMattention=soft2016.06 | 93.4 | — | |
| Zoneout2016.06 | 93.1 | — | |
| LSTM+Zoneout2019.10 | 93.1 | — | |
| LSTM# units=256, # params=270k2020.10 | 92.9 | — | |
| Recurrent Dropout2016.06 | 92.5 | — | |
| LSTM+Recurrent dropout2019.10 | 92.5 | — | |
| D-NTM discrete IBattention=discrete, reinforcement_baseline=IB2016.06 | 92.3 | — | |
| Unitary-RNN2016.06 | 91.4 | — | |
| uRNN2019.10 | 91.4 | — | |
| uRNN# units=512, # params=9k2020.10 | 91.4 | — | |
| NTM2016.06 | 90.9 | — | |
| LSTM2016.06 | 89.8 | — | |
| D-NTM discrete MABattention=discrete, reinforcement_baseline=MAB2016.06 | 89.6 | — | |
| LSTM2019.10 | 88 | — | |
| GRUNumber of Parameters=70K2021.02 | 87.3 | — | |
| LSTMNumber of Parameters=70K2021.02 | 85.7 | — | |
| I-RNN2016.06 | 82 | — | |
| IRNN2019.10 | 82 | — | |
| IndRNNNumber of layers=62018.03 | — | 1.3 | |
| IRNN2018.03 | — | 18 | |
| LSTM2018.03 | — | 12 | |
| LSTM+Recurrent batchnormRecurrent batchnorm=true2018.03 | — | 4.6 | |
| LSTM+Recurrent batchnorm+ZoneoutRecurrent batchnorm=true, Zoneout=true2018.03 | — | 4.1 | |
| LSTM+Recurrent dropoutRecurrent dropout=true2018.03 | — | 7.5 | |
| LSTM+ZoneoutZoneout=true2018.03 | — | 6.9 | |
| uRNN2018.03 | — | 8.6 |