List your company
ResearchFormerly WizWand

Visual Question Answering on VQA (val)

79.54Overall AccuracyALICEMIND-MMU
13.60430.72247.8464.958Aug 8, 2018Mar 21, 2019Nov 2, 2019Jun 14, 2020Jan 26, 2021Sep 8, 2021Apr 22, 2022
Updated 8mo ago

Evaluation results

MethodLinks
ALICEMIND-MMU2021.1179.5492.4770.6372
Human2021.1178.6994.8778.7966.34
BaselineBase Model=CLIP-ViL, Training Protocol=Re-Pretrain, Training Data Scale=100%2022.0473.91———
MADBase Model=VILLA, Training Protocol=Only Finetune, Teacher Visual Encoder=CLIP-V, Teacher Text Encoder=CLIP-T, Training Data Scale=100%2022.0473.02———
MADBase Model=VILLA, Training Protocol=Only Finetune, Teacher Visual Encoder=CLIP-V, Teacher Text Encoder=ROBERTa, Training Data Scale=100%2022.0472.43———
BaselineBase Model=VILLA, Training Protocol=Only Finetune, Training Data Scale=100%2022.0472.11———
MADBase Model=VILLA, Training Protocol=Only Finetune, Teacher Visual Encoder=ViT, Teacher Text Encoder=CLIP-T, Training Data Scale=100%2022.0472.02———
MADBase Model=UNITER, Training Protocol=Only Finetune, Teacher Visual Encoder=CLIP-V, Teacher Text Encoder=CLIP-T, Training Data Scale=100%2022.0471.94———
MADBase Model=VL-BERT, Training Protocol=Only Finetune, Teacher Visual Encoder=CLIP-V, Teacher Text Encoder=CLIP-T, Training Data Scale=100%2022.0471.42———
BaselineBase Model=UNITER, Training Protocol=Only Finetune, Training Data Scale=100%2022.0471.26———
MDBase Model=VL-BERT, Training Protocol=Only Finetune, Teacher Visual Encoder=CLIP-V, Teacher Text Encoder=CLIP-T, Training Data Scale=100%2022.0470.22———
BaselineBase Model=VL-BERT, Training Protocol=Only Finetune, Training Data Scale=100%2022.0469.05———
MADBase Model=VILLA, Training Protocol=Only Finetune, Teacher Visual Encoder=CLIP-V, Teacher Text Encoder=CLIP-T, Training Data Scale=3%2022.0466.93———
BaselineBase Model=CLIP-ViL, Training Protocol=Re-Pretrain, Training Data Scale=3%2022.0466.84———
MADBase Model=VILLA, Training Protocol=Only Finetune, Teacher Visual Encoder=CLIP-V, Teacher Text Encoder=ROBERTa, Training Data Scale=3%2022.0466.23———
MADBase Model=VILLA, Training Protocol=Only Finetune, Teacher Visual Encoder=ViT, Teacher Text Encoder=CLIP-T, Training Data Scale=3%2022.0466.1———
MADBase Model=UNITER, Training Protocol=Only Finetune, Teacher Visual Encoder=CLIP-V, Teacher Text Encoder=CLIP-T, Training Data Scale=3%2022.0465.93———
BaselineBase Model=VILLA, Training Protocol=Only Finetune, Training Data Scale=3%2022.0465.75———
MADBase Model=VL-BERT, Training Protocol=Only Finetune, Teacher Visual Encoder=CLIP-V, Teacher Text Encoder=CLIP-T, Training Data Scale=3%2022.0465.71———
MDBase Model=VL-BERT, Training Protocol=Only Finetune, Teacher Visual Encoder=CLIP-V, Teacher Text Encoder=CLIP-T, Training Data Scale=3%2022.0464.43———
BaselineBase Model=UNITER, Training Protocol=Only Finetune, Training Data Scale=3%2022.0464.43———
BaselineBase Model=VL-BERT, Training Protocol=Only Finetune, Training Data Scale=3%2022.0463.29———
LfFBase Architecture=UpDown [3], Evaluation Protocol=VQA-CE2021.0463.26———
ESRBase Architecture=UpDown [3], Evaluation Protocol=VQA-CE2021.0463.03———
LMH + RMFEBase Architecture=UpDown [3], Evaluation Protocol=VQA-CE2021.0462.97———
RandImgBase Architecture=UpDown [3], Evaluation Protocol=VQA-CE2021.0462.87———
UpDownEvaluation Protocol=VQA-CE2021.0462.83———
QGHC+MUTAN2018.0860.13———
QGHC+concat2018.0859.8———
LMHBase Architecture=UpDown [3], Evaluation Protocol=VQA-CE2021.0459.74———
QGHC#parameters=5.4M2018.0859.24———
QGHC-1#parameters=2.1M2018.0858.88———
MUTAN+MLB#parameters=17.5M2018.0858.76———
SANVisual Features=grid features, Evaluation Protocol=VQA-CE2021.0458.35———
MUTAN#parameters=4.9M2018.0858.16———
MLB#parameters=7.7M2018.0857.91———
MCB#parameters=32M2018.0857.39———
Concat2018.0856.92———
RUBiBase Architecture=UpDown [3], Evaluation Protocol=VQA-CE2021.0455.82———
AdapterBase Model=VL-BERT, Training Protocol=Only Finetune, Teacher Visual Encoder=CLIP-V, Teacher Text Encoder=CLIP-T, Training Data Scale=100%2022.0451.34———
Question-OnlyEvaluation Protocol=VQA-CE2021.0449.2———
ShortcutsEvaluation Protocol=VQA-CE2021.0444.71———
MADBase Model=VILLA, Training Protocol=Only Finetune, Teacher Visual Encoder=CLIP-V, Teacher Text Encoder=CLIP-T, Training Data Scale=0.3%2022.0440.16———
MADBase Model=UNITER, Training Protocol=Only Finetune, Teacher Visual Encoder=CLIP-V, Teacher Text Encoder=CLIP-T, Training Data Scale=0.3%2022.0439.75———
MADBase Model=VILLA, Training Protocol=Only Finetune, Teacher Visual Encoder=CLIP-V, Teacher Text Encoder=ROBERTa, Training Data Scale=0.3%2022.0439.04———
BaselineBase Model=CLIP-ViL, Training Protocol=Re-Pretrain, Training Data Scale=0.3%2022.0439.01———
MADBase Model=VILLA, Training Protocol=Only Finetune, Teacher Visual Encoder=ViT, Teacher Text Encoder=CLIP-T, Training Data Scale=0.3%2022.0438.79———
AdapterBase Model=VL-BERT, Training Protocol=Only Finetune, Teacher Visual Encoder=CLIP-V, Teacher Text Encoder=CLIP-T, Training Data Scale=3%2022.0437.41———
BaselineBase Model=VILLA, Training Protocol=Only Finetune, Training Data Scale=0.3%2022.0437.18———
MADBase Model=VL-BERT, Training Protocol=Only Finetune, Teacher Visual Encoder=CLIP-V, Teacher Text Encoder=CLIP-T, Training Data Scale=0.3%2022.0437.12———
MDBase Model=VL-BERT, Training Protocol=Only Finetune, Teacher Visual Encoder=CLIP-V, Teacher Text Encoder=CLIP-T, Training Data Scale=0.3%2022.0436.83———
BaselineBase Model=UNITER, Training Protocol=Only Finetune, Training Data Scale=0.3%2022.0436.46———
BaselineBase Model=VL-BERT, Training Protocol=Only Finetune, Training Data Scale=0.3%2022.0435.33———
Image-OnlyEvaluation Protocol=VQA-CE2021.0424.39———
AdapterBase Model=VL-BERT, Training Protocol=Only Finetune, Teacher Visual Encoder=CLIP-V, Teacher Text Encoder=CLIP-T, Training Data Scale=0.3%2022.0416.14———