Visual Commonsense Reasoning (Q->A) on VCR Shortcut Mitigated Evaluation SM (test)
76.32Q->A Accuracy (100%)VILLA (MAD*) [ROBERTa/CLIP-V]
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| VILLA (MAD*) [ROBERTa/CLIP-V]Base (Student) Model=VILLA, Method=Multimodal Adaptive Distillation (MAD*), Teacher Visual Encoder=CLIP-V, Teacher Textual Encoder=ROBERTa2022.04 | 76.32 | — | 41.98 | 56.85 | |
| VILLA (MAD*)Base (Student) Model=VILLA, Method=Multimodal Adaptive Distillation (MAD*), Teacher Visual Encoder=CLIP-V, Teacher Textual Encoder=CLIP-T2022.04 | 76.01 | — | 41.97 | 55.2 | |
| VILLA (Baseline)Base (Student) Model=VILLA, Training Paradigm=Only Finetune2022.04 | 75.43 | — | 29.41 | 54.15 | |
| VILLA (MAD*) [ViT/CLIP-T]Base (Student) Model=VILLA, Method=Multimodal Adaptive Distillation (MAD*), Teacher Visual Encoder=ViT, Teacher Textual Encoder=CLIP-T2022.04 | 75.35 | 39.44 | — | 54.29 | |
| VL-BERT (MAD*)Base (Student) Model=VL-BERT, Method=Multimodal Adaptive Distillation (MAD*), Teacher Visual Encoder=CLIP-V, Teacher Textual Encoder=CLIP-T2022.04 | 74.55 | 26.31 | 39.27 | — | |
| UNITER (MAD*)Base (Student) Model=UNITER, Method=Multimodal Adaptive Distillation (MAD*), Teacher Visual Encoder=CLIP-V, Teacher Textual Encoder=CLIP-T2022.04 | 74.24 | 26.49 | 41.83 | 54.64 | |
| CLIP-ViLpTraining Paradigm=Re-Pretrain, Teacher Visual Encoder=CLIP-V2022.04 | 66.83 | 33.41 | — | 52.44 | |
| VL-BERT (MD*)Base (Student) Model=VL-BERT, Method=Multimodal Distillation (MD*), Teacher Visual Encoder=CLIP-V, Teacher Textual Encoder=CLIP-T2022.04 | 54.88 | — | — | 39.27 | |
| UNITER (MD*)Base (Student) Model=UNITER, Method=Multimodal Distillation (MD*), Teacher Visual Encoder=CLIP-V, Teacher Textual Encoder=CLIP-T2022.04 | 54.58 | — | 38.21 | — | |
| UNITER (Baseline)Base (Student) Model=UNITER, Training Paradigm=Only Finetune2022.04 | 52.72 | 24.21 | 28.43 | — | |
| Direct FinetuneTraining Paradigm=Only Finetune, Teacher Visual Encoder=CLIP-V, Teacher Textual Encoder=CLIP-T2022.04 | 52.06 | 58.3 | — | 34.93 | |
| VL-BERT (Baseline)Base (Student) Model=VL-BERT, Training Paradigm=Only Finetune2022.04 | 49.27 | 23.24 | 26.37 | — | |
| AdaptersTraining Paradigm=Only Finetune, Teacher Visual Encoder=CLIP-V, Teacher Textual Encoder=CLIP-T2022.04 | 35.48 | — | 33.35 | 35.02 |