Visual Question Answering on VQA Introspect v1.0 (test)
78.93AccuracySelective Decomposition
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Selective DecompositionVQA Model=BLIP-2 13B, Decomposer=FLAN-T5 (T), Decomposer Params=80M2023.10 | 78.93 | 3.46 | 8 | 11.81 | |
| Selective DecompositionVQA Model=BLIP-2 13B, Decomposer=FLAN-T5 (T), Decomposer Params=250M2023.10 | 78.93 | 3.31 | 7 | 15.81 | |
| Selective DecompositionVQA Model=BLIP-2 13B, Decomposer=FLAN-T5 (T), Decomposer Params=780M2023.10 | 78.93 | 3.46 | 8 | 11.81 | |
| Selective DecompositionVQA Model=BLIP-2 13B, Decomposer=FLAN-T5 (T), Decomposer Params=3B2023.10 | 78.93 | 3.73 | 8 | 11.81 | |
| Selective DecompositionVQA Model=BLIP-2 13B, Decomposer=Galactica (T*), Decomposer Params=7B2023.10 | 78.93 | 3.69 | 8 | 11.61 | |
| Selective DecompositionVQA Model=BLIP-2 13B, Decomposer=FLAN-T5 (T), Decomposer Params=11B2023.10 | 78.93 | 3.68 | 8 | 11.81 | |
| Selective DecompositionVQA Model=BLIP-2 13B, Decomposer=BLIP-2 (I+T), Decomposer Params=11B2023.10 | 78.93 | 3.81 | 8 | 11.81 | |
| Selective DecompositionVQA Model=BLIP-2 3B, Decomposer=FLAN-T5 (T), Decomposer Params=80M2023.10 | 78.64 | 3.23 | 8 | 10.64 | |
| Selective DecompositionVQA Model=BLIP-2 3B, Decomposer=FLAN-T5 (T), Decomposer Params=250M2023.10 | 78.64 | 3.1 | 7 | 13.24 | |
| Selective DecompositionVQA Model=BLIP-2 3B, Decomposer=FLAN-T5 (T), Decomposer Params=780M2023.10 | 78.64 | 3.18 | 8 | 10.64 | |
| Selective DecompositionVQA Model=BLIP-2 3B, Decomposer=FLAN-T5 (T), Decomposer Params=3B2023.10 | 78.64 | 3.5 | 16 | 5.87 | |
| Selective DecompositionVQA Model=BLIP-2 3B, Decomposer=BLIP-2 (I+T), Decomposer Params=3B2023.10 | 78.64 | 3.75 | 20 | 5.24 | |
| Selective DecompositionVQA Model=BLIP-2 3B, Decomposer=Galactica (T*), Decomposer Params=7B2023.10 | 78.64 | 3.37 | 8 | 10.64 | |
| Selective DecompositionVQA Model=BLIP-2 3B, Decomposer=FLAN-T5 (T), Decomposer Params=11B2023.10 | 78.64 | 3.6 | 14 | 6.27 |