Selective Visual Question Answering on Mixed ID/OOD (90% VQA v2 + 10% AdVQA) (test)
74.79AccuracyMaxProb
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| MaxProbVQA Model=OFA-Large, VQA Train Set=A+B2023.06 | 74.79 | 1.3 | 43.7 | 65.95 | 8.26 | 62.24 | 27.09 | -2.46 | |
| SelectorVQA Model=OFA-Large, VQA Train Set=A+B, Selection Train Set=A+B, Selection Targets=Self2023.06 | 74.79 | 22.68 | 50.27 | 69.27 | 7.32 | 63.03 | 33.5 | 4.92 | |
| SelectorVQA Model=OFA-Large, VQA Train Set=A+B, Selection Train Set=A+B, Selection Targets=LYP2023.06 | 74.79 | 25.38 | 51.07 | 69.74 | 7.17 | 63.41 | 34.85 | 10.34 | |
| MaxProbVQA Model=OFA-Large, VQA Train Set=A2023.06 | 74.56 | 4.76 | 44.53 | 66.06 | 8.21 | 61.9 | 28.2 | 0.21 | |
| SelectorVQA Model=OFA-Large, VQA Train Set=A, Selection Train Set=B, Selection Targets=Self2023.06 | 74.56 | 23.53 | 50.17 | 68.76 | 7.33 | 62.96 | 34.43 | 9.88 | |
| MaxProbVQA Model=OFA-Base, VQA Train Set=A+B2023.06 | 72 | 1.74 | 37.02 | 57.57 | 9.78 | 58.11 | 22.09 | 0.53 | |
| SelectorVQA Model=OFA-Base, VQA Train Set=A+B, Selection Train Set=A+B, Selection Targets=Self2023.06 | 72 | 19.72 | 42.7 | 60.84 | 8.9 | 58.9 | 28.05 | 2.88 | |
| SelectorVQA Model=OFA-Base, VQA Train Set=A+B, Selection Train Set=A+B, Selection Targets=LYP2023.06 | 72 | 21.58 | 44.09 | 61.69 | 8.74 | 59.11 | 28.79 | 10.88 | |
| MaxProbVQA Model=OFA-Base, VQA Train Set=A2023.06 | 71.59 | 0.01 | 36.07 | 56.49 | 10.1 | 57.49 | 23.15 | -0.34 | |
| SelectorVQA Model=OFA-Base, VQA Train Set=A, Selection Train Set=B, Selection Targets=Self2023.06 | 71.59 | 18.32 | 41.48 | 59.74 | 9.19 | 57.97 | 27.22 | 9.09 | |
| MaxProbVQA Model=CLIP-VIL, VQA Train Set=A+B2023.06 | 67.12 | 2.6 | 26.13 | 45.25 | 12.97 | 50.49 | 16.59 | -0.93 | |
| SelectorVQA Model=CLIP-VIL, VQA Train Set=A+B, Selection Train Set=A+B, Selection Targets=Self2023.06 | 67.12 | 2.97 | 26.7 | 46.19 | 12.8 | 50.89 | 18.19 | -0.65 | |
| SelectorVQA Model=CLIP-VIL, VQA Train Set=A+B, Selection Train Set=A+B, Selection Targets=LYP2023.06 | 67.12 | 15.22 | 32.58 | 49.18 | 11.9 | 51.43 | 22.09 | 7.12 | |
| MaxProbVQA Model=CLIP-VIL, VQA Train Set=A2023.06 | 66.35 | 0 | 24.16 | 43.53 | 13.55 | 49.12 | 14.39 | -4.64 | |
| SelectorVQA Model=CLIP-VIL, VQA Train Set=A, Selection Train Set=B, Selection Targets=Self2023.06 | 66.35 | 12.69 | 31.12 | 46.96 | 12.47 | 50.36 | 20.15 | 5.22 |