Visual Question Answering on VQA Karpathy (test)
77.9Overall AccuracyMETALM
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| METALMPrediction Mode=Generative Prediction, Model Size=base2022.06 | 77.9 | — | — | |
| SimVLMhugecategory=Discriminative2021.08 | 77.2 | 81 | 17.5 | |
| SimVLMhugecategory=Generative2021.08 | 77 | 79.9 | 30.3 | |
| SimVLMlargecategory=Discriminative2021.08 | 76.7 | 80.4 | 17.3 | |
| SimVLMlargecategory=Generative2021.08 | 76.5 | 79.5 | 29.6 | |
| SimVLMbasecategory=Discriminative2021.08 | 75.3 | 79 | 16.7 | |
| SimVLMbasecategory=Generative2021.08 | 75.2 | 78.3 | 25.8 | |
| UNITERPrediction Mode=Discriminative Prediction, Model Size=base2022.06 | 74.4 | — | — | |
| VL-BARTPrediction Mode=Generative Prediction, Model Size=base2022.06 | 72.1 | — | — | |
| VL-T5Prediction Mode=Generative Prediction, Model Size=base2022.06 | 71.4 | — | — | |
| UNITERcategory=Discriminative2021.08 | 70.5 | 74.4 | 10 | |
| VLKD-ViT-B/16Prediction Mode=Generative Prediction, Model Size=base2022.06 | 69.2 | — | — | |
| VL-BARTcategory=Generative2021.08 | 68.6 | 72.1 | 13.2 | |
| VL-T5category=Generative2021.08 | 67.9 | 71.4 | 13.1 | |
| VL-T5category=Discriminative2021.08 | 66.4 | 70.2 | 7.1 | |
| VL-BARTcategory=Discriminative2021.08 | 65.7 | 69.4 | 7 | |
| METALMPrediction Mode=Generative Prediction, Model Size=base2022.06 | 21.1 | — | — | |
| VLKD-ViT-B/16Prediction Mode=Generative Prediction, Model Size=base2022.06 | 18.6 | — | — | |
| VL-BARTPrediction Mode=Generative Prediction, Model Size=base2022.06 | 13.2 | — | — | |
| VL-T5Prediction Mode=Generative Prediction, Model Size=base2022.06 | 13.1 | — | — | |
| UNITERPrediction Mode=Discriminative Prediction, Model Size=base2022.06 | 10 | — | — |