Visual Dialog on VisDial v0.9 (test)
64.1MRRNMN
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| NMN2019.09 | 64.1 | 50.9 | 80.1 | 88.8 | 4.45 | |
| CoAtt-GANSetting=Discriminative2017.11 | 63.98 | 50.29 | 80.71 | 88.81 | 4.47 | |
| PDUN2019.09 | 63.4 | 49.2 | 81 | 90.5 | 4.03 | |
| ACE2019.09 | 62.9 | 49 | 80.5 | 89.3 | 4.32 | |
| HCIAE-NP-ATTSetting=Discriminative2017.11 | 62.22 | 48.48 | 78.75 | 87.59 | 4.81 | |
| HCIAE2019.09 | 62.2 | 48.4 | 78.7 | 87.5 | 4.81 | |
| AMEM2019.09 | 62.2 | 48.5 | 78.6 | 87.4 | 4.85 | |
| SF-I2019.09 | 62 | 48.1 | 78.6 | 87.5 | 4.79 | |
| AMEMSetting=Discriminative2017.11 | 61.6 | 47.74 | 78.04 | 86.84 | 4.99 | |
| MN-QIH-DEncoder=Memory Network (MN), Input=Question + Image + History (QIH), Decoder=Discriminative (D)2016.11 | 59.65 | 45.55 | 76.22 | 85.37 | 5.46 | |
| MNSetting=Discriminative2017.11 | 59.65 | 45.55 | 76.22 | 85.37 | 5.46 | |
| MN2019.09 | 59.6 | 45.5 | 76.2 | 85.3 | 5.46 | |
| ECE2019.09 | 59 | 44.3 | 76.1 | 85.9 | 5.51 | |
| HREA-QIH-DEncoder=Hierarchical Recurrent Encoder with Attention (HREA), Input=Question + Image + History (QIH), Decoder=Discriminative (D)2016.11 | 58.68 | 44.82 | 74.81 | 84.36 | 5.66 | |
| HREASetting=Discriminative2017.11 | 58.68 | 44.82 | 74.81 | 84.36 | 5.66 | |
| HRE2019.09 | 58.6 | 44.8 | 74.8 | 84.3 | 5.65 | |
| MN-QH-DEncoder=Memory Network (MN), Input=Question + History (QH), Decoder=Discriminative (D)2016.11 | 58.49 | 44.03 | 75.26 | 84.49 | 5.68 | |
| HRE-QIH-DEncoder=Hierarchical Recurrent Encoder (HRE), Input=Question + Image + History (QIH), Decoder=Discriminative (D)2016.11 | 58.46 | 44.67 | 74.5 | 84.22 | 5.72 | |
| HRESetting=Discriminative2017.11 | 58.46 | 44.67 | 74.5 | 84.22 | 5.72 | |
| LF-QIH-DEncoder=Late Fusion (LF), Input=Question + Image + History (QIH), Decoder=Discriminative (D)2016.11 | 58.07 | 43.82 | 74.68 | 84.07 | 5.78 | |
| LFSetting=Discriminative2017.11 | 58.07 | 43.82 | 74.68 | 84.07 | 5.78 | |
| LF2019.09 | 58 | 43.8 | 74.6 | 84 | 5.78 | |
| HieCoAtt-QI-DEncoder=HieCoAtt, Input=Question + Image (QI), Decoder=Discriminative (D)2016.11 | 57.88 | 43.51 | 74.49 | 83.96 | 5.84 | |
| HieCoAtt-QISetting=Discriminative2017.11 | 57.88 | 43.51 | 74.49 | 83.96 | 5.84 | |
| SAN-QI-DEncoder=SAN, Input=Question + Image (QI), Decoder=Discriminative (D)2016.11 | 57.64 | 43.44 | 74.26 | 83.72 | 5.88 | |
| SAN-QISetting=Discriminative2017.11 | 57.64 | 43.44 | 74.26 | 83.72 | 5.88 | |
| LF-QI-DEncoder=Late Fusion (LF), Input=Question + Image (QI), Decoder=Discriminative (D)2016.11 | 57.59 | 43.33 | 74.27 | 83.68 | 5.87 | |
| HRE-QH-DEncoder=Hierarchical Recurrent Encoder (HRE), Input=Question + History (QH), Decoder=Discriminative (D)2016.11 | 56.95 | 42.7 | 73.25 | 82.97 | 6.11 | |
| LF-QH-DEncoder=Late Fusion (LF), Input=Question + History (QH), Decoder=Discriminative (D)2016.11 | 55.78 | 41.75 | 71.45 | 80.94 | 6.74 | |
| CoAtt-GANintermediate reward (Rinte)=included, Teacher Forcing (TF)=included2017.11 | 55.78 | 46.1 | 65.69 | 71.74 | 14.43 | |
| LF-Q-DEncoder=Late Fusion (LF), Input=Question (Q), Decoder=Discriminative (D)2016.11 | 55.08 | 41.24 | 70.45 | 79.83 | 7.08 | |
| CoAtt-GANintermediate reward (Rinte)=included2017.11 | 55.06 | 45.56 | 65.16 | 71.07 | 15.3 | |
| CoAtt-GANintermediate reward (Rinte)=excluded2017.11 | 54.15 | 44.52 | 64.17 | 70.31 | 16.28 | |
| CoAtt-G-MLETraining=Maximum Likelihood Estimation2017.11 | 54.11 | 44.32 | 63.82 | 69.75 | 16.47 | |
| HCIAEHistory-Conditioned Image Attentive Encoder=true2017.11 | 53.86 | 44.06 | 63.55 | 69.24 | 16.01 | |
| MN-QIH-GEncoder=Memory Network (MN), Input=Question + Image + History (QIH), Decoder=Generative (G)2016.11 | 52.59 | 42.29 | 62.85 | 68.88 | 17.06 | |
| Memory Network (MN)2017.11 | 52.59 | 42.29 | 62.85 | 68.88 | 17.06 | |
| HREA-QIH-GEncoder=Hierarchical Recurrent Encoder with Attention (HREA), Input=Question + Image + History (QIH), Decoder=Generative (G)2016.11 | 52.42 | 42.28 | 62.33 | 68.17 | 16.79 | |
| HREAAttention=true2017.11 | 52.42 | 42.28 | 62.33 | 68.17 | 16.79 | |
| HRE-QIH-GEncoder=Hierarchical Recurrent Encoder (HRE), Input=Question + Image + History (QIH), Decoder=Generative (G)2016.11 | 52.37 | 42.29 | 62.18 | 67.92 | 17.07 | |
| HREHierarchical Recurrent Encoder=true2017.11 | 52.37 | 42.29 | 62.18 | 67.92 | 17.07 | |
| LF-QI-GEncoder=Late Fusion (LF), Input=Question + Image (QI), Decoder=Generative (G)2016.11 | 52.04 | 42.04 | 61.65 | 67.66 | 16.84 | |
| LF-QIH-GEncoder=Late Fusion (LF), Input=Question + Image + History (QIH), Decoder=Generative (G)2016.11 | 51.99 | 41.83 | 61.78 | 67.59 | 17.07 | |
| Late Fusion (LF)2017.11 | 51.99 | 41.83 | 61.78 | 67.59 | 17.07 | |
| MN-QH-GEncoder=Memory Network (MN), Input=Question + History (QH), Decoder=Generative (G)2016.11 | 51.15 | 40.42 | 61.57 | 67.44 | 17.74 | |
| HRE-QH-GEncoder=Hierarchical Recurrent Encoder (HRE), Input=Question + History (QH), Decoder=Generative (G)2016.11 | 51.02 | 40.15 | 61.59 | 67.36 | 17.47 | |
| LF-QH-GEncoder=Late Fusion (LF), Input=Question + History (QH), Decoder=Generative (G)2016.11 | 50.55 | 39.73 | 60.86 | 66.68 | 17.78 | |
| LF-Q-GEncoder=Late Fusion (LF), Input=Question (Q), Decoder=Generative (G)2016.11 | 50.48 | 39.78 | 60.58 | 66.33 | 17.89 | |
| NN-QModel Category=Baseline, Input=Question (Q)2016.11 | 45.7 | 35.93 | 54.07 | 60.26 | 18.93 | |
| NN-QIModel Category=Baseline, Input=Question + Image (QI)2016.11 | 42.74 | 33.13 | 50.83 | 58.69 | 19.62 | |
| NNNearest Neighbor=true2017.11 | 42.74 | 33.13 | 50.83 | 58.69 | 19.62 | |
| VisDial-GNNEM iteration usage=full, Number of iterations=32019.04 | 41.26 | 27.15 | 56.47 | 71.97 | 8.86 | |
| SF-QIH2019.04 | 40.6 | 26.76 | 55.17 | 70.39 | 9.32 | |
| VisDial-GNNGraph configuration=constant2019.04 | 40.25 | 26.08 | 55.3 | 70.83 | 9.24 | |
| VisDial-GNNEM iteration usage=none2019.04 | 39.77 | 25.69 | 54.52 | 70.33 | 9.38 | |
| Answer priorModel Category=Baseline, Encoder=LSTM2016.11 | 37.35 | 23.55 | 48.52 | 53.23 | 26.5 | |
| Answer Prior2017.11 | 37.35 | 23.55 | 48.52 | 53.23 | 26.5 | |
| SF-QI2019.04 | 30.21 | 17.38 | 42.32 | 57.16 | 14.03 |