Visual Question Answering on WinogroundVQA v1.0 (test)
46.5AccuracySelective Decomposition
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Selective DecompositionVQA Model=BLIP-2 13B, Decomposer=FLAN-T5 (T), Decomposer Params=80M2023.10 | 46.5 | 6.94 | 95 | 0.22 | |
| Selective DecompositionVQA Model=BLIP-2 13B, Decomposer=FLAN-T5 (T), Decomposer Params=250M2023.10 | 46.5 | 22.12 | 99 | 0.09 | |
| Selective DecompositionVQA Model=BLIP-2 13B, Decomposer=FLAN-T5 (T), Decomposer Params=780M2023.10 | 46.5 | 0.06 | 2 | 3.83 | |
| Selective DecompositionVQA Model=BLIP-2 13B, Decomposer=FLAN-T5 (T), Decomposer Params=3B2023.10 | 46.5 | 0.69 | 92 | 0.29 | |
| Selective DecompositionVQA Model=BLIP-2 13B, Decomposer=Galactica (T*), Decomposer Params=7B2023.10 | 46.5 | 4.06 | 99 | 0.09 | |
| Selective DecompositionVQA Model=BLIP-2 13B, Decomposer=FLAN-T5 (T), Decomposer Params=11B2023.10 | 46.5 | 0.06 | 14 | 1.64 | |
| Selective DecompositionVQA Model=BLIP-2 13B, Decomposer=BLIP-2 (I+T), Decomposer Params=11B2023.10 | 46.5 | 0 | 1 | 5.49 | |
| Selective DecompositionVQA Model=BLIP-2 3B, Decomposer=FLAN-T5 (T), Decomposer Params=80M2023.10 | 45.81 | 5.94 | 94 | 0.41 | |
| Selective DecompositionVQA Model=BLIP-2 3B, Decomposer=FLAN-T5 (T), Decomposer Params=250M2023.10 | 45.81 | 22 | 99 | 0.19 | |
| Selective DecompositionVQA Model=BLIP-2 3B, Decomposer=FLAN-T5 (T), Decomposer Params=780M2023.10 | 45.81 | 0 | 1 | 8.59 | |
| Selective DecompositionVQA Model=BLIP-2 3B, Decomposer=FLAN-T5 (T), Decomposer Params=3B2023.10 | 45.81 | 0 | 1 | 8.59 | |
| Selective DecompositionVQA Model=BLIP-2 3B, Decomposer=BLIP-2 (I+T), Decomposer Params=3B2023.10 | 45.81 | 0 | 1 | 8.59 | |
| Selective DecompositionVQA Model=BLIP-2 3B, Decomposer=Galactica (T*), Decomposer Params=7B2023.10 | 45.81 | 0.06 | 2 | 7.4 | |
| Selective DecompositionVQA Model=BLIP-2 3B, Decomposer=FLAN-T5 (T), Decomposer Params=11B2023.10 | 45.81 | 0 | 1 | 8.59 |