Visual Commonsense Reasoning (Q→A) on VCR (test)
61.49Q->A Accuracy (3%, 1000 SP/C)VILLA (MAD*) [ROBERTa/CLIP-V]
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| VILLA (MAD*) [ROBERTa/CLIP-V]Base (Student) Model=VILLA, Method=Multimodal Adaptive Distillation (MAD*), Teacher Visual Encoder=CLIP-V, Teacher Textual Encoder=ROBERTa2022.04 | 61.49 | — | 43.11 | 78.91 | |
| VILLA (MAD*)Base (Student) Model=VILLA, Method=Multimodal Adaptive Distillation (MAD*), Teacher Visual Encoder=CLIP-V, Teacher Textual Encoder=CLIP-T2022.04 | 60.93 | — | 42.95 | 78.83 | |
| UNITER (MAD*)Base (Student) Model=UNITER, Method=Multimodal Adaptive Distillation (MAD*), Teacher Visual Encoder=CLIP-V, Teacher Textual Encoder=CLIP-T2022.04 | 60.88 | 26.78 | 42.23 | 77.05 | |
| VL-BERT (MAD*)Base (Student) Model=VL-BERT, Method=Multimodal Adaptive Distillation (MAD*), Teacher Visual Encoder=CLIP-V, Teacher Textual Encoder=CLIP-T2022.04 | 58.98 | 26.8 | 40.43 | 77.61 | |
| VILLA (MAD*) [ViT/CLIP-T]Base (Student) Model=VILLA, Method=Multimodal Adaptive Distillation (MAD*), Teacher Visual Encoder=ViT, Teacher Textual Encoder=CLIP-T2022.04 | 58.38 | 40.73 | — | 78.59 | |
| VILLA (Baseline)Base (Student) Model=VILLA, Training Paradigm=Only Finetune2022.04 | 57.01 | — | 34.93 | 78.27 | |
| CLIP-ViLpTraining Paradigm=Re-Pretrain, Teacher Visual Encoder=CLIP-V2022.04 | 53.54 | 34.63 | — | 68.36 | |
| VL-BERT (MD*)Base (Student) Model=VL-BERT, Method=Multimodal Distillation (MD*), Teacher Visual Encoder=CLIP-V, Teacher Textual Encoder=CLIP-T2022.04 | 39.06 | 36.78 | — | 55.91 | |
| Direct FinetuneTraining Paradigm=Only Finetune, Teacher Visual Encoder=CLIP-V, Teacher Textual Encoder=CLIP-T2022.04 | 38.1 | 54.82 | 38.23 | 54.23 | |
| AdaptersTraining Paradigm=Only Finetune, Teacher Visual Encoder=CLIP-V, Teacher Textual Encoder=CLIP-T2022.04 | 36.34 | — | 34.41 | 36.42 | |
| UNITER (MD*)Base (Student) Model=UNITER, Method=Multimodal Distillation (MD*), Teacher Visual Encoder=CLIP-V, Teacher Textual Encoder=CLIP-T2022.04 | 35.02 | 39.43 | — | 57.64 | |
| VL-BERT (Baseline)Base (Student) Model=VL-BERT, Training Paradigm=Only Finetune2022.04 | 34.85 | 23.37 | 30.85 | 53.48 | |
| UNITER (Baseline)Base (Student) Model=UNITER, Training Paradigm=Only Finetune2022.04 | 33.35 | 24.78 | 31.43 | 54.24 |