Visual Question Answering on CLEVR 1.0 (test)
99.6Overall AccuracyLEFT
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| LEFTPre-defined programs=No2023.10 | 99.6 | — | — | — | — | — | |
| NSCLPre-defined programs=Yes2023.10 | 99.2 | — | — | — | — | — | |
| TbD-net + reg + hresRegularization=true, Feature Map Resolution=28x282018.03 | 99.1 | 99.2 | 97.6 | 99.4 | 99.5 | 99.6 | |
| MAC2018.03 | 98.9 | 99.5 | 97.1 | 99.1 | 99.5 | 99.5 | |
| MAC2018.03 | 98.9 | 99.5 | 97.2 | 99.4 | 99.3 | 99.5 | |
| MACPre-defined programs=No2023.10 | 98.9 | — | — | — | — | — | |
| TbD-netRegularization=false, Feature Map Resolution=14x142018.03 | 98.7 | 98.9 | 96.8 | 99.1 | 99.4 | 99.2 | |
| TbD-net + regRegularization=true, Feature Map Resolution=14x142018.03 | 98.5 | 98.9 | 96.5 | 99 | 99.3 | 99.1 | |
| DDRprogParameters=9M, Epochs=52, additional program supervision=true2018.03 | 98.3 | 98.8 | 96.5 | 98.4 | 99.1 | 99 | |
| DDRprog2018.03 | 98.3 | 98.8 | 96.5 | 98.4 | 99.1 | 99 | |
| FiLM2018.08 | 97.7 | 99.1 | 94.3 | 96.8 | 99.1 | 99.1 | |
| CNN+GRU+FiLMdata augmentation=false2018.03 | 97.7 | 99.1 | 94.3 | 96.8 | 99.1 | 99.1 | |
| FiLM/CBNParameters=>50M, Epochs=802018.03 | 97.6 | 99.2 | 94.5 | 93.8 | 99.2 | 99 | |
| CNN+GRU+FiLMdata augmentation=true2018.03 | 97.6 | 99.3 | 94.3 | 93.4 | 99.3 | 99.3 | |
| CNN + GRU + FiLM2018.03 | 97.6 | 99.2 | 94.5 | 93.8 | 99.2 | 99 | |
| IEPParameters=41M, Epochs=12, additional program supervision=true2018.03 | 96.9 | 97.1 | 92.7 | 98.7 | 98.1 | 98.8 | |
| PG+EEextra supervisory information=true, Program Labels=700K2018.03 | 96.9 | 97.1 | 92.7 | 98.7 | 98.1 | 98.9 | |
| PG + EEProgram Count=700k2018.03 | 96.9 | 97.1 | 92.7 | 98.7 | 98.1 | 98.9 | |
| RNParameters=500k, Epochs=10002018.03 | 95.5 | 97.8 | 90.1 | 93.6 | 97.9 | 97.1 | |
| CNN+LSTM+RNdata augmentation=true, training from raw pixels=true2018.03 | 95.5 | 97.8 | 90.1 | 93.6 | 97.9 | 97.1 | |
| CNN + LSTM + RN2018.03 | 95.5 | 97.8 | 90.1 | 93.6 | 97.1 | 97.9 | |
| PG+EEextra supervisory information=true, Program Labels=18K2018.03 | 95.4 | 97.3 | 90.1 | 96.5 | 97.4 | 98 | |
| Human2018.03 | 92.6 | 96.6 | 86.7 | 86.4 | 94.9 | 96 | |
| Human2018.08 | 92.6 | 96.6 | 86.7 | 79 | 97 | 98 | |
| Human2018.03 | 92.6 | 96.6 | 86.7 | 86.5 | 95 | 96 | |
| Human2018.03 | 92.6 | 96.6 | 86.7 | 86.4 | 95 | 96 | |
| N2NMN2018.03 | 88.8 | 85.7 | 68.5 | 84.9 | 90 | 88.8 | |
| PG+EEextra supervisory information=true, Program Labels=9K2018.03 | 88.6 | 89.7 | 79.7 | 79.1 | 92.6 | 96 | |
| QGHC2018.08 | 86.3 | 78.1 | 91.17 | 67.3 | 93.65 | 86.24 | |
| End-to-End NMNadditional program supervision=true2018.03 | 83.7 | 85.7 | 68.5 | 84.9 | 89.9 | 88.7 | |
| N2NMN2018.08 | 83.7 | 85.7 | 68.5 | 73.8 | 93.1 | 92.6 | |
| N2NMNextra supervisory information=true2018.03 | 83.7 | 85.7 | 68.5 | 84.9 | 90 | 88.7 | |
| CNN+LSTM+SA+MLPStacked Spatial Attention=true2018.03 | 73.2 | 77.9 | 59.7 | 75.1 | 80.9 | 70.8 | |
| CNN+LSTM+SA+MLP2018.03 | 73.2 | 77.9 | 59.7 | 75.1 | 80.9 | 70.8 | |
| NMN2018.08 | 72.1 | 79.3 | 52.5 | 61.2 | 84.2 | 74.4 | |
| NMN2018.03 | 72.1 | 79.3 | 52.5 | 72.7 | 79 | 78 | |
| CNN+LSTM+SAStacked Spatial Attention=true2018.03 | 69.8 | 68.4 | 57.5 | 67.7 | 87.7 | 52 | |
| CNN-LSTM + SA2018.08 | 68.5 | 71.1 | 52.2 | 60 | 87 | 55 | |
| CNN+LSTM2018.03 | 54.3 | 68.2 | 47.8 | 70.1 | 48.9 | 54.6 | |
| CNN-LSTM2018.08 | 52.3 | 65.2 | 43.7 | 57 | 59 | 48 | |
| CNN+LSTM2018.03 | 52.3 | 65.2 | 43.7 | 67.1 | 49.3 | 53 | |
| CNN-LSTM + MCB2018.08 | 51.4 | 63.4 | 42.1 | 57 | 59 | 52 | |
| LSTM2018.03 | 47 | 61.8 | 42.5 | 70 | 36.5 | 51.1 | |
| LSTM2018.03 | 46.8 | 61.1 | 41.7 | 69.8 | 36.8 | 51.8 | |
| Q-type mode2018.03 | 42.1 | 50.2 | 34.6 | 51.1 | 36.9 | 51.2 | |
| Q-type baseline2018.03 | 41.8 | 50.2 | 34.6 | 51 | 36 | 51.3 |