Visual Dialogue on VisDial v1.0 (test)
75.2NDCGPretrained FT SOTA
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Pretrained FT SOTAFT=true, Shot=123K2022.04 | 75.2 | — | — | — | — | — | |
| VDBERTFine-tuning on dense annotations=true2022.05 | 74.54 | 33.15 | 61.58 | 77.15 | 50.74 | 7.18 | |
| UTCFine-tuning on dense annotations=true2022.05 | 74.32 | 37.12 | 63.98 | 79.88 | 50.24 | 6.48 | |
| MCAFine-tuning on dense annotations=true2022.05 | 72.47 | 20.67 | 56.67 | 72.12 | 37.68 | 8.89 | |
| UTCPre-training=VQA + CC2022.05 | 64.6 | 55.73 | 84.93 | 93.08 | 68.7 | 3.32 | |
| VisDial-BERT2022.05 | 63.87 | 53.85 | 84.68 | 93.25 | 67.5 | 3.32 | |
| UTC2022.05 | 62.65 | 52.25 | 83.55 | 92.23 | 66.27 | 3.48 | |
| VDBERT2022.05 | 59.96 | 51.63 | 82.23 | 90.68 | 65.44 | 3.9 | |
| LTMI2022.05 | 59.03 | 50.2 | 80.68 | 90.35 | 64.08 | 4.05 | |
| DAN2022.05 | 57.59 | 49.63 | 79.75 | 89.35 | 63.2 | 4.3 | |
| Synergistic2022.05 | 57.32 | 47.9 | 80.43 | 89.95 | 62.2 | 4.17 | |
| FGA2022.05 | 56.9 | 52.75 | 82.92 | 91.07 | 66.2 | 3.8 | |
| FlamingoFT=false, Shot=42022.04 | 55.6 | — | — | — | — | — | |
| FlamingoFT=false, Shot=322022.04 | 55.6 | — | — | — | — | — | |
| RvA2022.05 | 55.59 | 49.03 | 80.4 | 89.83 | 63.03 | 4.18 | |
| GNN2022.05 | 52.82 | 47.33 | 77.98 | 87.83 | 61.37 | 4.57 | |
| FlamingoFT=false, Shot=02022.04 | 52 | — | — | — | — | — | |
| Flamingo-9BFT=false, Shot=42022.04 | 50.4 | — | — | — | — | — | |
| Flamingo-9BFT=false, Shot=322022.04 | 50.4 | — | — | — | — | — | |
| MN-Att2022.05 | 49.58 | 42.42 | 74 | 84.35 | 56.9 | 5.59 | |
| Flamingo-9BFT=false, Shot=02022.04 | 48 | — | — | — | — | — | |
| MN2022.05 | 47.5 | 40.98 | 72.3 | 83.3 | 55.49 | 5.92 | |
| Flamingo-3BFT=false, Shot=42022.04 | 47.3 | — | — | — | — | — | |
| Flamingo-3BFT=false, Shot=322022.04 | 47.3 | — | — | — | — | — | |
| Flamingo-3BFT=false, Shot=02022.04 | 46.1 | — | — | — | — | — | |
| LF2022.05 | 45.31 | 40.95 | 72.45 | 82.23 | 55.42 | 5.95 |