Visual Commonsense Reasoning on VCR (val)
82.57AccuracyVILLA
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| VILLAModel size=Large2020.06 | 82.57 | — | — | — | |
| UNITERModel size=Large2020.06 | 80.49 | — | — | — | |
| VILLAModel size=Base2020.06 | 78.78 | — | — | — | |
| KVL-BERT_LARGEModel Scale=Large2020.12 | 78.6 | — | — | — | |
| VILLAModel size=Large2020.06 | 78.45 | — | — | — | |
| VL-BERTModel size=Large2020.06 | 77.9 | — | — | — | |
| VL-BERT_LARGEModel Scale=Large2020.12 | 77.9 | — | — | — | |
| UNITERModel size=Large2020.06 | 77.22 | — | — | — | |
| UNITERModel size=Base2020.06 | 77.03 | — | — | — | |
| KVL-BERT_LARGEModel Scale=Large2020.12 | 76.3 | — | — | — | |
| B2T22020.12 | 76 | — | — | — | |
| VILLAModel size=Base2020.06 | 75.54 | — | — | — | |
| VL-BERTModel size=Large2020.06 | 75.5 | — | — | — | |
| VL-BERT_LARGEModel Scale=Large2020.12 | 75.5 | — | — | — | |
| UNITER_BASE*Model Scale=Base, Pre-training Strategy=one-stage2020.12 | 75.3 | — | — | — | |
| KVL-BERT_BASEModel Scale=Base2020.12 | 75.1 | — | — | — | |
| UNITERModel size=Base2020.06 | 74.56 | — | — | — | |
| Unicoder-VL2020.06 | 74.5 | — | — | — | |
| ViLBERT2020.12 | 74.5 | — | — | — | |
| Unicoder-VL2020.12 | 74.5 | — | — | — | |
| ViLBERT2020.06 | 74.47 | — | — | — | |
| 12-in-12020.06 | 74.4 | — | — | — | |
| VL-BERT_BASEModel Scale=Base2020.12 | 74.4 | — | — | — | |
| KVL-BERT_BASEModel Scale=Base2020.12 | 74 | — | — | — | |
| 12-in-12020.06 | 73.8 | — | — | — | |
| VL-BERT_BASEModel Scale=Base2020.12 | 73.8 | — | — | — | |
| VisualBERT2020.06 | 73.2 | — | — | — | |
| VisualBERT2020.12 | 73.2 | — | — | — | |
| UNITER_BASE*Model Scale=Base, Pre-training Strategy=one-stage2020.12 | 72.8 | — | — | — | |
| Unicoder-VL2020.06 | 72.6 | — | — | — | |
| Unicoder-VL2020.12 | 72.6 | — | — | — | |
| ViLBERT2020.06 | 72.42 | — | — | — | |
| ViLBERT2020.12 | 72.4 | — | — | — | |
| B2T22020.12 | 71.9 | — | — | — | |
| VisualBERT2020.06 | 70.8 | — | — | — | |
| VisualBERT2020.12 | 70.8 | — | — | — | |
| CCN2020.12 | 70.6 | — | — | — | |
| HGL2020.12 | 70.6 | — | — | — | |
| HGL2020.12 | 69.4 | — | — | — | |
| CCN2020.12 | 67.4 | — | — | — | |
| R2C2020.12 | 67.2 | — | — | — | |
| VILLAModel size=Large2020.06 | 65.18 | — | — | — | |
| R2C2020.12 | 63.8 | — | — | — | |
| UNITERModel size=Large2020.06 | 62.59 | — | — | — | |
| KVL-BERT_LARGEModel Scale=Large2020.12 | 60 | — | — | — | |
| VILLAModel size=Base2020.06 | 59.75 | — | — | — | |
| VL-BERTModel size=Large2020.06 | 58.9 | — | — | — | |
| VL-BERT_LARGEModel Scale=Large2020.12 | 58.9 | — | — | — | |
| UNITERModel size=Base2020.06 | 57.76 | — | — | — | |
| KVL-BERT_BASEModel Scale=Base2020.12 | 55.6 | — | — | — | |
| 12-in-12020.06 | 55.2 | — | — | — | |
| VL-BERT_BASEModel Scale=Base2020.12 | 55.2 | — | — | — | |
| B2T22020.12 | 54.9 | — | — | — | |
| UNITER_BASE*Model Scale=Base, Pre-training Strategy=one-stage2020.12 | 54.9 | — | — | — | |
| Unicoder-VL2020.12 | 54.5 | — | — | — | |
| Unicoder-VL2020.06 | 54.4 | — | — | — | |
| ViLBERT2020.06 | 54.04 | — | — | — | |
| ViLBERT2020.12 | 54 | — | — | — | |
| VisualBERT2020.06 | 52.2 | — | — | — | |
| VisualBERT2020.12 | 52.2 | — | — | — | |
| HGL2020.12 | 49.1 | — | — | — | |
| CCN2020.12 | 47.7 | — | — | — | |
| R2C2020.12 | 43.1 | — | — | — | |
| ALBEFSupervision=weakly supervised2022.05 | — | 71.9 | 74.5 | 54.1 | |
| ASMv2-13Btraining_setting=multi-task2024.02 | — | 87.8 | 88.8 | 78.4 | |
| ASMv2-13Bsingle-task fine-tuning=true2024.02 | — | 88.4 | 89.9 | 79.4 | |
| B2T2Model=Single2019.08 | — | 71.9 | 76 | 54.9 | |
| B2T22022.05 | — | 73.2 | 77.1 | 56.6 | |
| BaselineBackbone=CLIP ViT-B/16, Evaluation Protocol=Zero-shot2023.07 | — | 53.24 | 46.51 | — | |
| ERNIE-VIL-L2024.02 | — | 78.5 | 83.4 | 65.8 | |
| ERNIE-VIL-LCode=Y2023.07 | — | 78.52 | 83.37 | 65.81 | |
| GPT4RoI-7Bsingle-task fine-tuning=true2024.02 | — | 87.4 | 89.6 | 78.6 | |
| GPT4RoI-7BCode=Y, #Params=7B+2023.07 | — | 87.4 | 89.6 | 78.6 | |
| PEVL2022.05 | — | 75.1 | 76.4 | 57.8 | |
| R2CModel=Single2019.08 | — | 63.8 | 67.2 | 43.1 | |
| R2C2022.05 | — | 63.8 | 67.2 | 43.1 | |
| R2CEvaluation Protocol=Supervised2023.07 | — | 63.8 | 67.2 | — | |
| Random2023.07 | — | 25 | 25 | — | |
| TAB-VCR2022.05 | — | 69.9 | 72.2 | 50.6 | |
| Unicoder-VL2022.05 | — | 72.6 | 74.5 | 54.4 | |
| Unicoder-VL2024.02 | — | 72.6 | 74.5 | 54.5 | |
| Unicoder-VLCode=Y2023.07 | — | 72.6 | 74.5 | 54.5 | |
| UniFine-BaseBackbone=CLIP ViT-B/16, Evaluation Protocol=Zero-shot2023.07 | — | 54.97 | 50.72 | — | |
| UniFine-LargeBackbone=CLIP ViT-L/14@336px, Evaluation Protocol=Zero-shot2023.07 | — | 58.48 | 51.88 | — | |
| UNITER2022.05 | — | 74.6 | 77 | 57.8 | |
| ViLBERTModel=Single2019.08 | — | 72.4 | 74.5 | 54 | |
| ViLBERT2024.02 | — | 72.4 | 74.5 | 54 | |
| VILBERT2022.05 | — | 72.4 | 74.5 | 54 | |
| VILBERTCode=Y, #Params=221M2023.07 | — | 72.4 | 74.5 | 54 | |
| VILLA2024.02 | — | 78.5 | 82.6 | 65.2 | |
| VILLA-LCode=Y2023.07 | — | 78.45 | 82.57 | 65.18 | |
| VisualBERTModel=Single2019.08 | — | 70.8 | 73.2 | 52.2 | |
| VisualBERT2022.05 | — | 70.8 | 73.2 | 52.2 | |
| VL-BERTBackbone=BERT-Base, Pre-training=None2019.08 | — | 73.1 | 73.8 | 54.2 | |
| VL-BERTBackbone=BERT-Base, Pre-training=Full2019.08 | — | 73.8 | 74.4 | 55.2 | |
| VL-BERTBackbone=BERT-Large, Pre-training=Full2019.08 | — | 75.5 | 77.9 | 58.9 | |
| VL-BERT2022.05 | — | 73.8 | 74.4 | 55.2 | |
| VLBERT2024.02 | — | 75.5 | 77.9 | 58.9 | |
| VLBERT-LCode=Y, #Params=383M2023.07 | — | 75.5 | 77.9 | 58.9 |