Visual Question Answering on VQA 2.0 (test)
80.2AccuracyPretrained FT SOTA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Pretrained FT SOTAFT=true, Shot=10K2022.04 | 80.2 | — | |
| FlamingoFT=false, Shot=322022.04 | 67.6 | — | |
| XNMsexpert layout=no2018.12 | 67.5 | — | |
| Up-Downexpert layout=no2018.12 | 66.3 | — | |
| StackNMNexpert layout=no, reported_in_original_paper=true2018.12 | 64.1 | — | |
| N2NMNexpert layout=yes, reported_in_original_paper=true2018.12 | 63.3 | — | |
| FlamingoFT=false, Shot=42022.04 | 63.1 | — | |
| Flamingo-9BFT=false, Shot=322022.04 | 60.4 | — | |
| Marginal LikelihoodObjective=log E[p], Expectation calculation=Enum2018.07 | 60.33 | 1.69 | |
| Soft AttentionObjective=log p(y | E[z]), Expectation calculation=N/A2018.07 | 58.93 | 1.76 | |
| Variational AttentionObjective=Eq [log p] - KL, Expectation calculation=Enum2018.07 | 58.44 | 1.69 | |
| Hard AttentionObjective=Ep [log p], Expectation calculation=Enum2018.07 | 57.6 | 1.78 | |
| Variational AttentionObjective=Eq [log p] - KL, Expectation calculation=Sample2018.07 | 57.52 | 1.75 | |
| Flamingo-3BFT=false, Shot=322022.04 | 57.1 | — | |
| Hard AttentionObjective=Ep [log p], Expectation calculation=Sample2018.07 | 56.3 | 1.82 | |
| Flamingo-9BFT=false, Shot=42022.04 | 56.3 | — | |
| FlamingoFT=false, Shot=02022.04 | 56.3 | — | |
| Flamingo-3BFT=false, Shot=42022.04 | 53.2 | — | |
| Flamingo-9BFT=false, Shot=02022.04 | 51.8 | — | |
| Flamingo-3BFT=false, Shot=02022.04 | 49.2 | — | |
| METALMevaluation protocol=zero-shot2022.06 | 41.1 | — | |
| VLKD_ViT-B/16evaluation protocol=zero-shot2022.06 | 38.6 | — | |
| Zero/Few shot SOTAFT=false, Shot=162022.04 | 38.2 | — | |
| Frozenevaluation protocol=zero-shot2022.06 | 29.5 | — |