Visual Entailment on SNLI-VE (val)
85Overall AccuracySOHO
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| SOHOBackbone=R1012021.04 | 85 | — | — | — | |
| ALBEF (14M)Pre-training dataset=>10M images, Model size=Base size2022.10 | 84.2 | — | — | — | |
| MAPPre-training dataset=<10M images, Model size=Base size2022.10 | 81.4 | — | — | — | |
| UNIMOModel Size=large2020.12 | 81.11 | — | — | — | |
| SoftMask++#Img=4M, Encoder Type=Detector-Free (D.F.)2023.04 | 80.9 | — | — | — | |
| METERPre-training dataset=<10M images, Model size=Base size2022.10 | 80.86 | — | — | — | |
| ALBEFPre-training Images=14M2021.07 | 80.8 | — | — | — | |
| SimVLM-BasePre-training dataset=>10M images, Model size=Base size2022.10 | 80.8 | — | — | — | |
| MADBase Model=VILLA, Teacher Model (VE)=CLIP-V, Teacher Model (TE)=CLIP-T, Training Samples=Full2022.04 | 80.67 | — | — | — | |
| MADBase Model=VILLA, Teacher Model (TE)=RoBERTa, Training Samples=Full2022.04 | 80.64 | — | — | — | |
| BaselineBase Model=CLIP-ViLp, Teacher Model (VE)=CLIP-V, Training Samples=Full2022.04 | 80.61 | — | — | — | |
| Knowledge-CLIPmode=Fine-tuning2022.10 | 80.52 | — | — | — | |
| TCL#Images=4M2022.02 | 80.51 | — | — | — | |
| TCL#Img=4M, Encoder Type=Detector-Free (D.F.)2023.04 | 80.5 | — | — | — | |
| CODIS#Img=4M, Encoder Type=Detector-Free (D.F.)2023.04 | 80.5 | — | — | — | |
| VILLAModel size=Large2020.06 | 80.18 | — | — | — | |
| VillaModel Size=large2020.12 | 80.18 | — | — | — | |
| ALBEFPre-training Images=4M2021.07 | 80.14 | — | — | — | |
| ALBEF#Images=4M2022.02 | 80.14 | — | — | — | |
| MADBase Model=UNITER, Teacher Model (VE)=CLIP-V, Teacher Model (TE)=CLIP-T, Training Samples=Full2022.04 | 80.14 | — | — | — | |
| ALBEFmode=Fine-tuning2022.10 | 80.14 | — | — | — | |
| ALBEF (4M)Pre-training dataset=<10M images, Model size=Base size2022.10 | 80.14 | — | — | — | |
| ALBEF#Img=4M, Encoder Type=Detector-Free (D.F.)2023.04 | 80.1 | — | — | — | |
| MDBase Model=UNITER, Teacher Model (VE)=CLIP-V, Teacher Model (TE)=CLIP-T, Training Samples=Full2022.04 | 80.08 | — | — | — | |
| UNIMO#Images=4M2022.02 | 80 | — | — | — | |
| UNIMO#Img=4M, Encoder Type=Object Detector (O.D.)2023.04 | 80 | — | — | — | |
| UNIMO-BasePre-training dataset=<10M images, Model size=Base size2022.10 | 80 | — | — | — | |
| UNIMOModel Size=base2020.12 | 80 | — | — | — | |
| Deep Feature Separation LossVision Encoder=ViT-B/16, Text Encoder=BERT-base2023.03 | 79.95 | — | — | — | |
| Brownian Bridge LossVision Encoder=ViT-B/16, Text Encoder=BERT-base2023.03 | 79.86 | — | — | — | |
| Geometric Consistency LossVision Encoder=ViT-B/16, Text Encoder=BERT-base2023.03 | 79.82 | — | — | — | |
| ALBEFVision Encoder=ViT-B/16, Text Encoder=BERT-base2023.03 | 79.69 | — | — | — | |
| BaselineBase Model=VILLA, Training Samples=Full2022.04 | 79.64 | — | — | — | |
| CLIPmode=Fine-tuning2022.10 | 79.51 | — | — | — | |
| VILLA#Img=4M, Encoder Type=Object Detector (O.D.)2023.04 | 79.5 | — | — | — | |
| VILLAModel size=Base2020.06 | 79.47 | — | — | — | |
| VILLA2021.07 | 79.47 | — | — | — | |
| VILLA#Images=4M2022.02 | 79.47 | — | — | — | |
| VILLAmode=Fine-tuning2022.10 | 79.47 | — | — | — | |
| VILLA2023.03 | 79.47 | — | — | — | |
| VillaModel Size=base2020.12 | 79.47 | — | — | — | |
| CODISVision Encoder=ViT-B/16, Text Encoder=BERT-base2023.03 | 79.45 | — | — | — | |
| UNITERModel size=Large2020.06 | 79.39 | — | — | — | |
| UNITERModel Size=large2020.12 | 79.39 | — | — | — | |
| ALBEF LORAParams=644, TFLOPS=7.142023.10 | 79.34 | — | — | — | |
| ALBEF BaseParams=581, TFLOPS=7.052023.10 | 79.29 | — | — | — | |
| VICHA2023.03 | 79.2 | — | — | — | |
| BaselineBase Model=UNITER, Training Samples=Full2022.04 | 79.02 | — | — | — | |
| FLAVAmode=Fine-tuning2022.10 | 78.89 | — | — | — | |
| UNITER#Img=4M, Encoder Type=Object Detector (O.D.)2023.04 | 78.6 | — | — | — | |
| UNITERModel size=Base2020.06 | 78.59 | — | — | — | |
| UNITERBackbone=R1012021.04 | 78.59 | — | — | — | |
| UNITER2021.07 | 78.59 | — | — | — | |
| UNITER#Images=4M2022.02 | 78.59 | — | — | — | |
| UNITERmode=Fine-tuning2022.10 | 78.59 | — | — | — | |
| UNITER2023.03 | 78.59 | — | — | — | |
| UNITER-BasePre-training dataset=<10M images, Model size=Base size2022.10 | 78.59 | — | — | — | |
| UNITERParams=116, TFLOPS=0.372023.10 | 78.59 | — | — | — | |
| UNITERModel Size=base2020.12 | 78.59 | — | — | — | |
| ALBEF PELAParams=259, TFLOPS=2.472023.10 | 78.55 | — | — | — | |
| MADBase Model=VILLA, Teacher Model (VE)=ViT, Teacher Model (TE)=CLIP-T, Training Samples=Full2022.04 | 78.37 | — | — | — | |
| ALBEF ToMeParams=581, TFLOPS=2.552023.10 | 77.58 | — | — | — | |
| 12-in-12023.10 | 76.59 | — | — | — | |
| MADBase Model=VL-BERT, Teacher Model (VE)=CLIP-V, Teacher Model (TE)=CLIP-T, Training Samples=Full2022.04 | 75.75 | — | — | — | |
| MDBase Model=VL-BERT, Teacher Model (VE)=CLIP-V, Teacher Model (TE)=CLIP-T, Training Samples=Full2022.04 | 75.08 | — | — | — | |
| BaselineBase Model=VL-BERT, Training Samples=Full2022.04 | 74.66 | — | — | — | |
| ALBEF TinyBERTParams=392, TFLOPS=4.552023.10 | 73.83 | — | — | — | |
| ALBEF MaskAlignParams=392, TFLOPS=4.552023.10 | 73.74 | — | — | — | |
| modality relation distillation paradigm2021.12 | 72.66 | — | — | — | |
| EVE-Image*Input type=Image features2019.01 | 71.56 | 71.04 | 70.55 | 73.1 | |
| EVE-ImageBackbone=R1012021.04 | 71.56 | — | — | — | |
| EVE-ImageEvaluation Protocol=Supervised2023.07 | 71.56 | 71.04 | 70.55 | 73.1 | |
| KD2021.12 | 71.43 | — | — | — | |
| EVE-ROI*Input type=Region of Interest (ROI) features2019.01 | 70.81 | 68.55 | 68.78 | 75.1 | |
| Attention Top-DownImage features=Pre-trained CNN feature maps2019.01 | 70.53 | 70.23 | 68.66 | 72.71 | |
| Attention Bottom-UpImage features=Top 10 ROIs from Mask-RCNN2019.01 | 69.34 | 71.26 | 70.1 | 66.67 | |
| MADBase Model=VILLA, Teacher Model (VE)=CLIP-V, Teacher Model (TE)=CLIP-T, Training Samples=30,0002022.04 | 68.43 | — | — | — | |
| MADBase Model=UNITER, Teacher Model (VE)=CLIP-V, Teacher Model (TE)=CLIP-T, Training Samples=30,0002022.04 | 68.34 | — | — | — | |
| Image CaptioningImage Encoder=ResNet152, Caption Decoder=LSTM2019.01 | 67.83 | 66.61 | 69.23 | 67.65 | |
| CLIPRes50x16Training=Image + Text, Evaluation=Text + Text2022.03 | 67.64 | — | — | — | |
| Relational Network2019.01 | 67.56 | 67.86 | 67.8 | 67.02 | |
| CLIPRes50x16Training=Text + Text, Evaluation=Image + Text2022.03 | 67.24 | — | — | — | |
| Direct FinetuneBase Model=VL-BERT, Training Samples=Full2022.04 | 66.91 | — | — | — | |
| Hypothesis Only2019.01 | 66.68 | 67.54 | 66.9 | 65.6 | |
| CLIPViT-B/16Training=Image + Text, Evaluation=Text + Text2022.03 | 65.97 | — | — | — | |
| CLIPRes101Training=Image + Text, Evaluation=Text + Text2022.03 | 65.67 | — | — | — | |
| MADBase Model=VL-BERT, Teacher Model (VE)=CLIP-V, Teacher Model (TE)=CLIP-T, Training Samples=30,0002022.04 | 65.37 | — | — | — | |
| CLIPRes101Training=Text + Text, Evaluation=Image + Text2022.03 | 64.29 | — | — | — | |
| CLIPViT-B/16Training=Text + Text, Evaluation=Image + Text2022.03 | 64.11 | — | — | — | |
| MADBase Model=VILLA, Teacher Model (VE)=CLIP-V, Teacher Model (TE)=CLIP-T, Training Samples=3,0002022.04 | 59.65 | — | — | — | |
| MADBase Model=UNITER, Teacher Model (VE)=CLIP-V, Teacher Model (TE)=CLIP-T, Training Samples=3,0002022.04 | 59.42 | — | — | — | |
| Direct FinetuneBase Model=VL-BERT, Training Samples=30,0002022.04 | 57.02 | — | — | — | |
| MADBase Model=VL-BERT, Teacher Model (VE)=CLIP-V, Teacher Model (TE)=CLIP-T, Training Samples=3,0002022.04 | 56.78 | — | — | — | |
| Direct FinetuneBase Model=VL-BERT, Training Samples=3,0002022.04 | 50.91 | — | — | — | |
| UniFine-LargeBackbone=CLIP ViT-L/14@336px, Evaluation Protocol=Zero-shot2023.07 | 50.16 | 68.29 | 29.57 | 52.68 | |
| UniFine-BaseBackbone=CLIP ViT-B/16, Evaluation Protocol=Zero-shot2023.07 | 49.41 | 68.08 | 28.55 | 51.67 | |
| BaselineBackbone=CLIP ViT-B/16, Evaluation Protocol=Zero-shot2023.07 | 47.37 | 67.59 | 18.66 | 55.92 | |
| Direct FinetuneBase Model=VL-BERT, Training Samples=0-shot2022.04 | 47.23 | — | — | — | |
| AdapterBase Model=VL-BERT, Teacher Model (VE)=CLIP-V, Teacher Model (TE)=CLIP-T, Training Samples=30,0002022.04 | 41.75 | — | — | — | |
| AdapterBase Model=VL-BERT, Teacher Model (VE)=CLIP-V, Teacher Model (TE)=CLIP-T, Training Samples=3,0002022.04 | 41.65 | — | — | — |