Visual Entailment on SNLI-VE
0.91AccuracyOFA
Evaluation Results
| Method | Links | |
|---|---|---|
| OFAEvaluation Protocol=Supervised2023.05 | 0.91 | |
| SimVLMbasepublic data=false2021.12 | 0.842 | |
| UNITEREvaluation Protocol=Supervised2023.05 | 0.794 | |
| FLAVApublic data=true2021.12 | 0.79 | |
| UNITERbasepublic data=true2021.12 | 0.783 | |
| UNITERInteraction=Fine-grained vision-language interaction2022.09 | 0.783 | |
| VisualBERTpublic data=true2021.12 | 0.773 | |
| ERNIE-ViL 2.0Interaction=Shallow cross-modal interaction, Model Type=English pre-trained2022.09 | 0.766 | |
| ViLBERTpublic data=true2021.12 | 0.757 | |
| CLIP-ViT-B/16public data=false2021.12 | 0.74 | |
| CLIP-ViT-B/16 (PMD)public data=true, pretraining dataset=PMD2021.12 | 0.735 | |
| UniTpublic data=true, config=shared, (COCO init.)2021.12 | 0.731 | |
| EVE-ImageEvaluation Protocol=Supervised2023.05 | 0.716 | |
| IdealGPTEvaluation Protocol=Zero-Shot2023.05 | 0.553 | |
| VLATTACKPre-trained Model=OFA, Attack Modality=Multimodality2023.10 | 0.4178 | |
| BLIP-2 ViT-G + LSKDEvaluation Protocol=Zero-shot, Backbone=ViT-G2023.12 | 0.403 | |
| LLaVAEvaluation Protocol=Zero-Shot2023.05 | 0.403 | |
| CLIP ViT-B-16*Evaluation Protocol=Zero-shot, Backbone=ViT-B-162023.12 | 0.36 | |
| MiniGPT4Evaluation Protocol=Zero-Shot2023.05 | 0.351 | |
| BLIP ViT-LEvaluation Protocol=Zero-shot, Backbone=ViT-L2023.12 | 0.342 | |
| BLIP-2 ViT-GEvaluation Protocol=Zero-shot, Backbone=ViT-G2023.12 | 0.334 | |
| Random Guess2023.05 | 0.333 | |
| CLIP ViT-L-14x336Evaluation Protocol=Zero-shot, Backbone=ViT-L-14x3362023.12 | 0.319 | |
| BLIP-2 ViT-LEvaluation Protocol=Zero-shot, Backbone=ViT-L2023.12 | 0.317 | |
| BSAPre-trained Model=OFA, Attack Modality=Image Only2023.10 | 0.2919 | |
| FDAPre-trained Model=OFA, Attack Modality=Image Only2023.10 | 0.209 | |
| Co-AttackPre-trained Model=OFA, Attack Modality=Multimodality2023.10 | 0.1866 | |
| SSPPre-trained Model=OFA, Attack Modality=Image Only2023.10 | 0.1511 | |
| DRPre-trained Model=OFA, Attack Modality=Image Only2023.10 | 0.1371 | |
| BERT-Attack (B&A)Pre-trained Model=OFA, Attack Modality=Text Only2023.10 | 0.1051 | |
| R&RPre-trained Model=OFA, Attack Modality=Text Only2023.10 | 0.0492 |