Selective Visual Question Answering on Mixed ID/OOD 50% VQA v2 / 50% AdVQA (test)
63.02AccuracyMaxProb
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| MaxProbVQA Model=OFA-Large, VQA Train Set=A, Selection Train Set=-, Selection Targets=-2023.06 | 63.02 | 0.31 | 11.53 | 27.85 | 17.18 | 43.42 | -3.11 | -34.01 | |
| SelectorVQA Model=OFA-Large, VQA Train Set=A, Selection Train Set=B, Selection Targets=Self2023.06 | 63.01 | 5.56 | 20.11 | 35.51 | 15.52 | 45.49 | 6.48 | -26.57 | |
| Selector (LYP-A Self-B)VQA Model=OFA-Large, VQA Train Set=A, Selection Train Set=A+B, Selection Targets=LYP-A Self-B2023.06 | 63.01 | 5.07 | 19.65 | 34.8 | 15.61 | 45.55 | 6.11 | -18.23 | |
| MaxProbVQA Model=OFA-Large, VQA Train Set=A+B, Selection Train Set=-, Selection Targets=-2023.06 | 62.93 | 0.12 | 6.22 | 26.58 | 17.58 | 43.4 | -6.02 | -40.93 | |
| SelectorVQA Model=OFA-Large, VQA Train Set=A+B, Selection Train Set=A+B, Selection Targets=Self2023.06 | 62.93 | 1 | 18.55 | 33.48 | 16.03 | 44.14 | 2.57 | -43.03 | |
| Selector (LYP)VQA Model=OFA-Large, VQA Train Set=A+B, Selection Train Set=A+B, Selection Targets=LYP2023.06 | 62.93 | 3.51 | 19.74 | 34.18 | 15.78 | 45.03 | 4.19 | -29.46 | |
| SelectorVQA Model=OFA-Base, VQA Train Set=A+B, Selection Train Set=A+B, Selection Targets=Self2023.06 | 59.62 | 2.29 | 15.78 | 27.38 | 18.7 | 39.88 | -0.74 | -36.1 | |
| Selector (LYP)VQA Model=OFA-Base, VQA Train Set=A+B, Selection Train Set=A+B, Selection Targets=LYP2023.06 | 59.62 | 3.98 | 17.13 | 28.53 | 18.3 | 40.35 | -0.49 | -11.27 | |
| MaxProbVQA Model=OFA-Base, VQA Train Set=A+B, Selection Train Set=-, Selection Targets=-2023.06 | 59.61 | 0.06 | 6.91 | 20.86 | 20.17 | 38.45 | -12.19 | -31.48 | |
| SelectorVQA Model=OFA-Base, VQA Train Set=A, Selection Train Set=B, Selection Targets=Self2023.06 | 59.18 | 3.21 | 15.98 | 26.27 | 19 | 38.49 | 0.1 | -11.07 | |
| Selector (LYP-A Self-B)VQA Model=OFA-Base, VQA Train Set=A, Selection Train Set=A+B, Selection Targets=LYP-A Self-B2023.06 | 59.18 | 2.28 | 15.38 | 26.72 | 18.8 | 39.24 | 3.46 | -19.92 | |
| MaxProbVQA Model=OFA-Base, VQA Train Set=A, Selection Train Set=-, Selection Targets=-2023.06 | 59.17 | 0.01 | 5.78 | 18.48 | 20.8 | 38.14 | -6.11 | -29.98 | |
| MaxProbVQA Model=CLIP-ViL, VQA Train Set=A+B, Selection Train Set=-, Selection Targets=-2023.06 | 53.83 | 0.97 | 3.66 | 12.27 | 25.23 | 29.82 | -6.4 | -15.61 | |
| SelectorVQA Model=CLIP-ViL, VQA Train Set=A+B, Selection Train Set=A+B, Selection Targets=Self2023.06 | 53.83 | 0.04 | 5.52 | 13.38 | 24.96 | 30.82 | -2.96 | -11.5 | |
| Selector (LYP)VQA Model=CLIP-ViL, VQA Train Set=A+B, Selection Train Set=A+B, Selection Targets=LYP2023.06 | 53.83 | 3.41 | 11.19 | 20.42 | 23.22 | 31.85 | 5.49 | -0.04 | |
| MaxProbVQA Model=CLIP-ViL, VQA Train Set=A, Selection Train Set=-, Selection Targets=-2023.06 | 52.66 | 0 | 3.08 | 9.77 | 26.57 | 27.65 | -13.2 | -20.6 | |
| SelectorVQA Model=CLIP-ViL, VQA Train Set=A, Selection Train Set=B, Selection Targets=Self2023.06 | 52.66 | 4.19 | 10.29 | 18.17 | 24.49 | 30.62 | -2.24 | -7.94 |