Open-Ended Visual Question Answering on VQA 1.0 (test-dev)

66.7Overall AccuracyEnsemble of 7 Att. models

Evaluation Results

Method
Ensemble of 7 Att. models2016.06	66.7	83.4	39.8	58.5
Dual-MFA2017.11	66.01	83.59	40.18	56.84
RelAtt2018.05	65.69	83.55	36.92	56.94
MCB + Att. + GloVe + Genome2016.06	65.4	82.3	37.2	57.4
Dual-MLB2017.11	65.12	83.32	39.96	55.26
MCB + Att. + Genome2016.06	65.1	81.7	38.2	57
Naver Labs (challenge 2nd)2016.06	64.9	83.5	39.8	54.8
MLB2017.11	64.89	84.13	37.85	54.57
MCB + Att. + GloVe2016.06	64.7	82.5	37.6	55.6
Fukui et al. 2016 ResNet*2016.06	64.7	82.5	37.6	55.6
MCB2017.11	64.7	82.5	37.6	55.6
MLAN2018.05	64.6	83.8	40.2	53.7
MLB2018.05	64.53	83.41	37.82	54.43
DAN2016.11	64.3	83	39.1	53.9
MCB2018.05	64.2	82.2	37.7	54.8
MCB + Att.2016.06	64.2	82.2	37.7	54.8
MCB2016.11	64.2	82.2	37.7	54.8
Ours ResNet2016.06	63.3	81.9	39	53
RAU2016.11	63.3	81.9	39	53
VQA-Machine2017.11	63.1	81.5	38.4	53
MCB + Genome2016.06	62.3	81.7	36.6	51.5
DAN2016.11	62	82.1	38.2	50.2
HieCoAtt2016.06	61.8	79.7	38.7	51.7
Lu et al. 2016 ResNet*2016.06	61.8	79.7	38.7	51.7
HieCoAtt2016.11	61.8	79.7	38.7	51.7
HieCoAtt2017.11	61.8	79.7	38.7	51.7
Kim et al. 2016 ResNet*2016.06	61.7	82.3	38.8	49.3
MRN2016.11	61.7	82.3	38.8	49.3
MRN2016.06	61.68	82.28	38.82	49.25
MRN2018.05	61.68	82.28	38.82	49.25
MRN2017.11	61.68	82.28	38.82	49.25
MRN2016.06	61.47	82.28	39.09	48.76
MRN2016.06	61.45	82.36	38.4	48.81
Ours_FULL2016.06	61.3	81.5	37	49.6
Ours_MS2016.06	61	81.3	36.8	49.1
MCB2016.06	60.8	81.2	35.1	49.3
MRN2016.06	60.77	82.1	39.11	47.46
QRU2018.05	60.72	82.29	37.02	47.67
QRU2017.11	60.72	82.29	37.02	47.67
MRN2016.06	60.68	82.4	38.69	47.1
MRN2016.06	60.53	82.53	38.34	46.78
DMN+2016.06	60.3	80.5	36.8	48.3
DMN+2018.05	60.3	80.5	36.8	48.3
DMN+2016.06	60.3	80.5	36.8	48.3
Xiong, Merity, and Socher 20162016.06	60.3	80.5	36.8	48.3
DMN+2016.11	60.3	80.5	36.8	48.3
DMN+2017.11	60.3	80.5	36.8	48.3
D-NMN2016.06	59.4	81.1	38.6	45.5
FDA2016.06	59.24	81.14	36.16	45.77
FDA2018.05	59.24	81.14	36.16	45.77
FDA2017.11	59.24	81.14	36.16	45.77
FDA2016.06	59.2	81.1	36.2	45.8
AMA2016.06	59.2	81	38.4	45.2
Wu et al. 20162016.06	59.2	81	38.4	45.2
ACK2016.11	59.2	81	38.4	45.2
ACK2016.06	59.17	81.01	38.42	45.23
AMA2018.05	59.17	81.01	38.42	45.23
Ours_SS2016.06	59	78.3	37.1	47.6
SAN2016.06	58.7	79.3	36.6	46.1
SAN2018.05	58.7	79.3	36.6	46.1
SAN2016.06	58.7	79.3	36.6	46.1
Yang et al. 20162016.06	58.7	79.3	36.6	46.1
SAN2016.11	58.7	79.3	36.6	46.1
SAN2017.11	58.7	79.3	36.6	46.1
NMN2016.06	58.6	81.2	38	44
NMN2016.11	58.6	81.2	38	44
AYN2016.06	58.4	78.4	36.4	46.3
Deep Q+I2016.06	58.02	80.87	36.46	43.4
SMem2016.06	58	80.9	37.3	43.1
SMem2018.05	57.99	80.87	37.32	43.12
SMem2017.11	57.99	80.87	37.32	43.12
D-NMN2016.06	57.9	80.5	37.4	43.1
Andreas et al. 2016b2016.06	57.9	80.5	37.4	43.1
VQA team2016.06	57.8	80.5	36.8	43.1
Antol et al. 20152016.06	57.8	80.5	36.8	43.1
VQA team2016.11	57.8	80.5	36.8	43.1
V2L2018.05	57.46	78.9	36.11	40.07
Andreas et al. 2016a2016.06	57.3	79.7	37.3	39.3
DPPnet2016.06	57.22	80.71	37.24	41.69
DPPnet2018.05	57.22	80.71	37.24	41.69
DPPnet2017.11	57.22	80.71	37.24	41.69
DPPnet2016.06	57.2	80.7	37.2	41.7
Noh, Seo, and Han 20162016.06	57.2	80.7	37.2	41.7
DPPnet2016.11	57.2	80.7	37.2	41.7
iBOWING2017.11	55.72	76.55	35.03	42.62
iBOWIMG2016.06	55.7	76.5	35	42.6
Zhou et al. 20152016.06	55.7	76.6	35	42.6
iBOWIMG2016.11	55.7	76.5	35	42.6
LSTM Q+I2016.06	53.74	78.94	35.24	36.42
LSTM Q+I2018.05	53.74	78.94	35.24	36.42
LSTM Q+I2017.11	53.74	78.94	35.24	36.42
LSTM Q+I (Antol et al. 2015)2016.06	53.7	78.9	35.2	36.4
Q+I2016.06	52.64	75.55	33.67	37.37
Bow Q+I (Antol et al. 2015)2016.06	52.6	75.6	33.7	37.4
LSTM Q (Antol et al. 2015)2016.06	48.8	78.2	35.7	26.6
LSTM Q2016.06	48.76	78.2	35.68	26.59
BOW Q (Antol et al. 2015)2016.06	48.1	75.7	36.7	27.1
Question2016.06	48.09	75.66	36.7	27.14
Image2016.06	28.13	64.01	0.42	3.77
I (Antol et al. 2015)2016.06	28.1	64	0.4	3.8