Selective Visual Question Answering on Mixed ID/OOD 66.7% VQA v2 / 33.3% AdVQA (test)
67.79AccSelector
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| SelectorVQA Model=OFA-Large, VQA Train Set=A, Selection Train Set=B, Selection Targets=Self2023.06 | 67.79 | 9.18 | 32.42 | 50.06 | 11.6 | 52.74 | 18.31 | -12.07 | |
| Selector (LYP-A Self-B)VQA Model=OFA-Large, VQA Train Set=A, Selection Train Set=A+B, Selection Targets=LYP-A Self-B2023.06 | 67.79 | 11.24 | 31.01 | 50.37 | 11.65 | 52.76 | 18.47 | -6.53 | |
| MaxProbVQA Model=OFA-Large, VQA Train Set=A, Selection Train Set=-, Selection Targets=-2023.06 | 67.78 | 0.39 | 22.67 | 43.82 | 13.05 | 50.88 | 10.05 | -20.68 | |
| MaxProbVQA Model=OFA-Large, VQA Train Set=A+B, Selection Train Set=-, Selection Targets=-2023.06 | 67.78 | 0.13 | 21.01 | 42.31 | 13.37 | 51.02 | 7.58 | -26.6 | |
| SelectorVQA Model=OFA-Large, VQA Train Set=A+B, Selection Train Set=A+B, Selection Targets=Self2023.06 | 67.77 | 6.58 | 30.15 | 48.83 | 11.98 | 51.79 | 15.37 | -23.92 | |
| Selector (LYP)VQA Model=OFA-Large, VQA Train Set=A+B, Selection Train Set=A+B, Selection Targets=LYP2023.06 | 67.77 | 9.44 | 30.87 | 49.69 | 11.75 | 52.51 | 16.95 | -12.91 | |
| MaxProbVQA Model=OFA-Base, VQA Train Set=A+B, Selection Train Set=-, Selection Targets=-2023.06 | 64.63 | 0.03 | 17.57 | 33.94 | 15.43 | 46.32 | 2.11 | -19.21 | |
| SelectorVQA Model=OFA-Base, VQA Train Set=A+B, Selection Train Set=A+B, Selection Targets=Self2023.06 | 64.63 | 5.11 | 25.83 | 40.13 | 14.09 | 47.58 | 10.75 | -21.18 | |
| Selector (LYP)VQA Model=OFA-Base, VQA Train Set=A+B, Selection Train Set=A+B, Selection Targets=LYP2023.06 | 64.63 | 9.41 | 27.89 | 42 | 13.8 | 48.03 | 11.89 | -2.81 | |
| MaxProbVQA Model=OFA-Base, VQA Train Set=A, Selection Train Set=-, Selection Targets=-2023.06 | 64.17 | 0.01 | 18.83 | 34.15 | 15.71 | 46.05 | 5.33 | -28.66 | |
| SelectorVQA Model=OFA-Base, VQA Train Set=A, Selection Train Set=B, Selection Targets=Self2023.06 | 64.17 | 6.97 | 25.19 | 39.53 | 14.44 | 46.41 | 11.98 | -3.47 | |
| Selector (LYP-A Self-B)VQA Model=OFA-Base, VQA Train Set=A, Selection Train Set=A+B, Selection Targets=LYP-A Self-B2023.06 | 64.16 | 6.76 | 25.39 | 41.49 | 14.26 | 46.99 | 14.42 | -9.89 | |
| MaxProbVQA Model=CLIP-ViL, VQA Train Set=A+B, Selection Train Set=-, Selection Targets=-2023.06 | 59.29 | 1.11 | 10.17 | 24.99 | 19.58 | 38.42 | 2.99 | -9.79 | |
| SelectorVQA Model=CLIP-ViL, VQA Train Set=A+B, Selection Train Set=A+B, Selection Targets=Self2023.06 | 59.29 | 0.07 | 11.21 | 25.86 | 19.28 | 39.17 | 5.9 | -7.37 | |
| Selector (LYP)VQA Model=CLIP-ViL, VQA Train Set=A+B, Selection Train Set=A+B, Selection Targets=LYP2023.06 | 59.29 | 7.07 | 19.13 | 31.53 | 17.94 | 39.85 | 12.67 | 3.4 | |
| MaxProbVQA Model=CLIP-ViL, VQA Train Set=A, Selection Train Set=-, Selection Targets=-2023.06 | 58.36 | 0 | 7.08 | 21.97 | 20.62 | 36.59 | -1.47 | -14.38 | |
| SelectorVQA Model=CLIP-ViL, VQA Train Set=A, Selection Train Set=B, Selection Targets=Self2023.06 | 58.36 | 5.87 | 17.41 | 29.21 | 18.9 | 38.76 | 7.11 | -2.2 |