Video Prediction on Moving MNIST (test)
15.67MSEMogaNet
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| MogaNetParameters=46.8 M, FLOPs=16.5 G, FPS=255 s, Epochs=2000, Framework=SimVP2022.11 | 15.67 | 51.84 | 0.9661 | — | — | — | |
| VANParameters=44.5 M, FLOPs=16.0 G, FPS=288 s, Epochs=2000, Framework=SimVP2022.11 | 16.21 | 53.57 | 0.9646 | — | — | — | |
| HorNetParameters=45.7 M, FLOPs=16.3 G, FPS=287 s, Epochs=2000, Framework=SimVP2022.11 | 17.4 | 55.7 | 0.9624 | — | — | — | |
| ConvNextParameters=37.3 M, FLOPs=14.1 G, FPS=344 s, Epochs=2000, Framework=SimVP2022.11 | 17.58 | 55.76 | 0.9617 | — | — | — | |
| UniformerParameters=44.8 M, FLOPs=16.5 G, FPS=296 s, Epochs=2000, Framework=SimVP2022.11 | 18.01 | 57.52 | 0.9609 | — | — | — | |
| MGP-VAEk=1, Loss function=geodesic loss2020.01 | 18.5 | — | — | 185.1 | — | — | |
| MLP-MixerParameters=38.2 M, FLOPs=14.7 G, FPS=334 s, Epochs=2000, Framework=SimVP2022.11 | 18.85 | 59.86 | 0.9589 | — | — | — | |
| GMG2025.03 | 19.0741 | 60.7413 | 0.9586 | — | — | 24.4606 | |
| SwinParameters=46.1 M, FLOPs=16.4 G, FPS=294 s, Epochs=2000, Framework=SimVP2022.11 | 19.11 | 59.84 | 0.9584 | — | — | — | |
| Swin-LSTM2025.03 | 19.4554 | 61.2669 | 0.9571 | — | — | 24.3593 | |
| ViTParameters=46.1 M, FLOPs=16.9 G, FPS=290 s, Epochs=2000, Framework=SimVP2022.11 | 19.74 | 61.65 | 0.9539 | — | — | — | |
| TAU2025.03 | 19.9112 | 62.1182 | 0.9562 | — | — | 24.3096 | |
| SimVP-VAN2025.03 | 20.5918 | 63.5674 | 0.9547 | — | — | 24.1267 | |
| PoolformerParameters=37.1 M, FLOPs=14.1 G, FPS=341 s, Epochs=2000, Framework=SimVP2022.11 | 20.96 | 64.31 | 0.9539 | — | — | — | |
| SimVPParameters=58.0 M, FLOPs=19.4 G, FPS=209 s, Epochs=2000, Framework=SimVP2022.11 | 21.15 | 64.15 | 0.9536 | — | — | — | |
| WasT2025.03 | 22.0719 | 70.8779 | 0.9491 | — | — | 23.7451 | |
| ConvMixerParameters=3.9 M, FLOPs=5.5 G, FPS=658 s, Epochs=2000, Framework=SimVP2022.11 | 22.3 | 67.37 | 0.9507 | — | — | — | |
| CrevNetreproducibility=reproduced by official code, Memory=224 MB, FLOPs=1.652 G, Training time=≈ 10d (300k iters)2022.06 | 22.3 | — | 0.949 | — | — | — | |
| SimVP-gSTA2025.03 | 22.5268 | 67.8671 | 0.95 | — | — | 23.6783 | |
| SimVPreproducibility=reproduced by official code, Memory=412 MB, FLOPs=1.676 G, Training time=≈ 2d (2k epochs)2022.06 | 23.8 | — | 0.948 | — | — | — | |
| PhyDNetreproducibility=reproduced by official code, Memory=200 MB, FLOPs=1.633 G, Training time=≈ 10d (2k epochs)2022.06 | 24.4 | — | 0.947 | — | — | — | |
| PredRNNParameters (M)=38.62024.05 | 24.53 | 73.12 | 0.9462 | — | — | — | |
| ARKMParameters (M)=3.82024.05 | 24.83 | 74.83 | 0.9423 | — | — | — | |
| ARKM (w/o H1)Parameters (M)=3.82024.05 | 25.19 | 75.74 | 0.9401 | — | — | — | |
| MGP-VAEk=1, Loss function=standard2020.01 | 25.4 | — | — | 198.4 | — | — | |
| SimVP-Poolformer2025.03 | 25.5146 | 74.6528 | 0.9429 | — | — | 23.1193 | |
| MogaNetParameters=46.8 M, FLOPs=16.5 G, FPS=255 s, Epochs=200, Framework=SimVP2022.11 | 25.57 | 75.19 | 0.9429 | — | — | — | |
| VANParameters=44.5 M, FLOPs=16.0 G, FPS=288 s, Epochs=200, Framework=SimVP2022.11 | 26.1 | 76.11 | 0.9417 | — | — | — | |
| SimVP-ViT2025.03 | 26.4819 | 77.5663 | 0.9371 | — | — | 23.0279 | |
| MAUParameters (M)=4.52024.05 | 26.86 | 78.22 | 0.9398 | — | — | — | |
| ConvNextParameters=37.3 M, FLOPs=14.1 G, FPS=344 s, Epochs=200, Framework=SimVP2022.11 | 26.94 | 77.23 | 0.9397 | — | — | — | |
| SwinLSTMParameters (M)=20.12024.05 | 27.74 | 77.21 | 0.9313 | — | — | — | |
| PhyDNetParameters (M)=3.12024.05 | 28.19 | 78.64 | 0.9374 | — | — | — | |
| MLP-MixerParameters=38.2 M, FLOPs=14.7 G, FPS=334 s, Epochs=200, Framework=SimVP2022.11 | 29.52 | 83.36 | 0.9338 | — | — | — | |
| HorNetParameters=45.7 M, FLOPs=16.3 G, FPS=287 s, Epochs=200, Framework=SimVP2022.11 | 29.64 | 83.26 | 0.9331 | — | — | — | |
| SwinParameters=46.1 M, FLOPs=16.4 G, FPS=294 s, Epochs=200, Framework=SimVP2022.11 | 29.7 | 84.05 | 0.9331 | — | — | — | |
| ConvLSTMParameters (M)=15.02024.05 | 29.8 | 90.64 | 0.9288 | — | — | — | |
| CrevNetParameters (M)=5.02024.05 | 30.15 | 86.28 | 0.935 | — | — | — | |
| UniformerParameters=44.8 M, FLOPs=16.5 G, FPS=296 s, Epochs=200, Framework=SimVP2022.11 | 30.38 | 85.87 | 0.9308 | — | — | — | |
| PoolformerParameters=37.1 M, FLOPs=14.1 G, FPS=341 s, Epochs=200, Framework=SimVP2022.11 | 31.79 | 88.48 | 0.9271 | — | — | — | |
| ConvMixerParameters=3.9 M, FLOPs=5.5 G, FPS=658 s, Epochs=200, Framework=SimVP2022.11 | 32.09 | 88.93 | 0.9259 | — | — | — | |
| SimVPParameters=58.0 M, FLOPs=19.4 G, FPS=209 s, Epochs=200, Framework=SimVP2022.11 | 32.15 | 89.05 | 0.9268 | — | — | — | |
| SimVPParameters (M)=58.02024.05 | 32.15 | 89.05 | 0.9268 | — | — | — | |
| ViTParameters=46.1 M, FLOPs=16.9 G, FPS=290 s, Epochs=200, Framework=SimVP2022.11 | 35.15 | 95.87 | 0.9139 | — | — | — | |
| DDPAEk=12020.01 | 35.2 | — | — | 201.6 | — | — | |
| E3D-LSTMParameters (M)=51.02024.05 | 35.97 | 78.28 | 0.932 | — | — | — | |
| E3D-LSTMMemory=2695 MB, FLOPs=381.3 G2022.06 | 41.3 | — | 0.92 | — | — | — | |
| MIM*Base structure=Causal LSTM2018.11 | 44.2 | 101.1 | 0.91 | — | — | — | |
| DRNetk=12020.01 | 45.2 | — | — | 236.7 | — | — | |
| Causal LSTM2018.11 | 46.5 | 106.8 | 0.898 | — | — | — | |
| CausalLSTMMemory=2017 MB, FLOPs=106.8 G2022.06 | 46.5 | — | 0.898 | — | — | — | |
| MCnetk=12020.01 | 50.1 | — | — | 248.2 | — | — | |
| MIMBase structure=ST-LSTM2018.11 | 52 | 116.5 | 0.874 | — | — | — | |
| PredRNN2018.11 | 56.8 | 126.1 | 0.867 | — | — | — | |
| PredRNNMemory=1666 MB, FLOPs=192.9 G2022.06 | 56.8 | — | 0.867 | — | — | — | |
| Grathwohl, Wilsonk=12020.01 | 59.3 | — | — | 291.2 | — | — | |
| VPN baseline2018.11 | 64.1 | 131 | 0.87 | — | — | — | |
| MGP-VAEk=2, Loss function=geodesic loss2020.01 | 69.2 | — | — | 531.4 | — | — | |
| FRNN2018.11 | 69.7 | 150.3 | 0.813 | — | — | — | |
| MGP-VAEk=2, Loss function=standard2020.01 | 72.2 | — | — | 554.2 | — | — | |
| DDPAEk=22020.01 | 75.6 | — | — | 556.2 | — | — | |
| DRNetk=22020.01 | 86.3 | — | — | 586.7 | — | — | |
| DFN2018.11 | 89 | 172.8 | 0.726 | — | — | — | |
| PredRNN++description=Causal LSTM + GHU (Final)2018.04 | 91.1 | — | 0.733 | — | — | — | |
| MCnetk=22020.01 | 91.1 | — | — | 595.5 | — | — | |
| CDNA2018.11 | 97.4 | 175.3 | 0.721 | — | — | — | |
| PredRNN + GHU2018.04 | 98.4 | — | 0.713 | — | — | — | |
| Causal LSTM2018.04 | 100.7 | — | 0.685 | — | — | — | |
| ConvLSTM2018.11 | 103.3 | 182.9 | 0.707 | — | — | — | |
| ConvLSTMMemory=1043MB, FLOPs=107.4G2022.06 | 103.3 | — | 0.707 | — | — | — | |
| Causal LSTMvariant=spatial-to-temporal2018.04 | 103.6 | — | 0.672 | — | — | — | |
| TrajGRU2018.11 | 106.9 | 190.1 | 0.713 | — | — | — | |
| PredRNN2018.04 | 112.2 | — | 0.645 | — | — | — | |
| Grathwohl, Wilsonk=22020.01 | 112.3 | — | — | 657.2 | — | — | |
| FC-LSTM2018.11 | 118.3 | 209.4 | 0.69 | — | — | — | |
| DMVFNParameters (M)=3.52024.05 | 123.67 | 179.96 | 0.814 | — | — | — | |
| VPNreproduced=true2018.04 | 129.6 | — | 0.62 | — | — | — | |
| CDNA2018.04 | 142.3 | — | 0.609 | — | — | — | |
| DFN2018.04 | 149.5 | — | 0.601 | — | — | — | |
| ConvLSTM2018.04 | 156.2 | — | 0.597 | — | — | — | |
| TrajGRU2018.04 | 163 | — | 0.588 | — | — | — | |
| FC-LSTM2018.04 | 180.1 | — | 0.583 | — | — | — | |
| Baseline modelDescription=Simple baseline architecture with 8 RMBs in encoders and 12 RMBs in decoders2016.10 | — | — | — | — | 110.1 | — | |
| Brabandere et al.2016.10 | — | — | — | — | 285.2 | — | |
| Conv-LSTMNumber of parameters=7,585,2962016.05 | — | — | — | 367.1 | — | — | |
| DFNNumber of parameters=637,3612016.05 | — | — | — | 285.2 | — | — | |
| FC-LSTMNumber of parameters=142,667,7762016.05 | — | — | — | 341.2 | — | — | |
| Lower bound2019.06 | — | — | — | — | 85.1 | — | |
| Lower Bound2016.10 | — | — | — | — | 86.3 | — | |
| Patraucean et al.2016.10 | — | — | — | — | 179.8 | — | |
| Shi et al.2016.10 | — | — | — | — | 367.2 | — | |
| Srivastava et al.2016.10 | — | — | — | — | 341.2 | — | |
| Subscale Video TransformerSubscaling=Spatial2019.06 | — | — | — | — | 91.8 | — | |
| Subscale Video TransformerSubscaling=Spatiotemporal2019.06 | — | — | — | — | 90 | — | |
| Video TransformerSubscaling=None2019.06 | — | — | — | — | 86.2 | — | |
| VPNDescription=Video Pixel Network with 8 RMBs in encoders and 12 RMBs in decoders2016.10 | — | — | — | — | 87.6 | — | |
| VPN2019.06 | — | — | — | — | 87.6 | — |