Visual Dialog on VisDial
62.27MRRAMEM+SEQ-QI
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| AMEM+SEQ-QIHistory embedding (+H)=false, Attention mechanism (ATT)=true, Number of parameters=1.7 M (0.7x)2017.09 | 62.27 | 48.53 | 78.66 | 87.43 | 4.86 | |
| AMEM+SEQ-QIHHistory embedding (+H)=true, Attention mechanism (ATT)=true, Number of parameters=2.3 M (1.0x)2017.09 | 62.1 | 48.4 | 78.39 | 87.12 | 4.92 | |
| AMEM-QIHistory embedding (+H)=false, Attention mechanism (ATT)=true, Number of parameters=1.7 M (0.7x)2017.09 | 61.96 | 48.24 | 78.33 | 87.11 | 4.92 | |
| AMEM-QIHHistory embedding (+H)=true, Attention mechanism (ATT)=true, Number of parameters=2.3 M (1.0x)2017.09 | 61.92 | 48.05 | 78.39 | 87.12 | 4.88 | |
| MN-QIHHistory embedding (+H)=true, Attention mechanism (ATT)=true, Number of parameters=14.7 M (6.4x)2017.09 | 59.65 | 45.55 | 76.22 | 85.37 | 5.46 | |
| HREA-QIHHistory embedding (+H)=true, Attention mechanism (ATT)=true, Number of parameters=16.8 M (7.3x)2017.09 | 58.68 | 44.82 | 74.81 | 84.36 | 5.66 | |
| MN-QHHistory embedding (+H)=true, Attention mechanism (ATT)=false, Number of parameters=12.4 M (5.4x)2017.09 | 58.49 | 44.03 | 75.26 | 84.49 | 5.68 | |
| HRE-QIHHistory embedding (+H)=true, Attention mechanism (ATT)=true, Number of parameters=16.8 M (7.3x)2017.09 | 58.46 | 44.67 | 74.5 | 84.22 | 5.72 | |
| LF-QIHHistory embedding (+H)=true, Attention mechanism (ATT)=true, Number of parameters=14.5 M (6.3x)2017.09 | 58.07 | 43.82 | 74.68 | 84.07 | 5.78 | |
| HieCoAtt-QIHistory embedding (+H)=false, Attention mechanism (ATT)=true, Number of parameters=n/a2017.09 | 57.88 | 43.51 | 74.49 | 83.96 | 5.84 | |
| SAN-QIHistory embedding (+H)=false, Attention mechanism (ATT)=true, Number of parameters=n/a2017.09 | 57.64 | 43.44 | 74.26 | 83.72 | 5.88 | |
| LF-QIHistory embedding (+H)=false, Attention mechanism (ATT)=true, Number of parameters=10.4 M (4.6x)2017.09 | 57.59 | 43.33 | 74.27 | 83.68 | 5.87 | |
| HRE-QHHistory embedding (+H)=true, Attention mechanism (ATT)=false, Number of parameters=15.0 M (6.5x)2017.09 | 56.95 | 42.7 | 73.25 | 82.97 | 6.11 | |
| LF-QHHistory embedding (+H)=true, Attention mechanism (ATT)=false, Number of parameters=12.4 M (5.4x)2017.09 | 55.78 | 41.75 | 71.45 | 80.94 | 6.74 | |
| LF-QHistory embedding (+H)=false, Attention mechanism (ATT)=false, Number of parameters=8.3 M (3.6x)2017.09 | 55.08 | 41.24 | 70.45 | 79.83 | 7.08 | |
| LION-12Bparameters=12B2023.11 | 50.42 | — | — | — | — | |
| InstructBLIP (T5XXL)backbone=T5XXL2023.11 | 50.41 | — | — | — | — | |
| LION-4Bparameters=4B2023.11 | 50.02 | — | — | — | — | |
| InstructBLIP (T5XL)backbone=T5XL2023.11 | 48.75 | — | — | — | — | |
| InstructBLIPzero-shot=true, backbone=FlanT5-XXL2023.05 | 48.5 | — | — | — | — | |
| InstructBLIP+ (T5XXL)backbone=T5XXL, uses_in-house_data=true2023.11 | 48.5 | — | — | — | — | |
| Flamingo-9Bparameters=9B2023.11 | 48 | — | — | — | — | |
| BLIP-2zero-shot=true, backbone=FlanT5-XXL2023.05 | 46.9 | — | — | — | — | |
| InstructBLIPzero-shot=true, backbone=FlanT5-XL2023.05 | 46.6 | — | — | — | — | |
| InstructBLIP+ (T5XL)backbone=T5XL, uses_in-house_data=true2023.11 | 46.6 | — | — | — | — | |
| Flamingo-3Bparameters=3B2023.11 | 46.1 | — | — | — | — | |
| BLIP-2zero-shot=true, backbone=FlanT5-XL2023.05 | 45.7 | — | — | — | — | |
| BLIVA2023.11 | 45.63 | — | — | — | — | |
| InstructBLIPzero-shot=true, backbone=Vicuna-13B2023.05 | 45.4 | — | — | — | — | |
| InstructBLIPzero-shot=true, backbone=Vicuna-7B2023.05 | 45.2 | — | — | — | — | |
| BLIP-2zero-shot=true, backbone=Vicuna-13B2023.05 | 45.1 | — | — | — | — | |
| BLIP-2zero-shot=true, backbone=Vicuna-7B2023.05 | 44.9 | — | — | — | — | |
| Answer priorHistory embedding (+H)=false, Attention mechanism (ATT)=false, Number of parameters=n/a2017.09 | 37.35 | 23.55 | 48.52 | 53.23 | 26.5 | |
| MiniGPT4evaluated_by_authors=true2023.11 | 16.52 | — | — | — | — | |
| LLaVAevaluated_by_authors=true2023.11 | 8.65 | — | — | — | — | |
| MiniGPTV2evaluated_by_authors=true2023.11 | 8.47 | — | — | — | — |