Visual Question Answering on VQAv2 (Accuracy and Average)
54.1AccuracySI
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SILVLM=Open-Flamingo, Language Model=MPT-7B, Shots=16-shot2023.12 | 54.1 | — | |
| SILVLM=Open-Flamingo, Language Model=MPT-7B, Shots=8-shot2023.12 | 52.95 | — | |
| SQLVLM=Open-Flamingo, Language Model=LLAMA-7B, Shots=16-shot2023.12 | 51.88 | — | |
| SILVLM=Open-Flamingo, Language Model=LLAMA-7B, Shots=16-shot2023.12 | 51.7 | — | |
| RSLVLM=Open-Flamingo, Language Model=MPT-7B, Shots=8-shot2023.12 | 51.05 | — | |
| RSLVLM=Open-Flamingo, Language Model=MPT-7B, Shots=16-shot2023.12 | 50.89 | — | |
| SQLVLM=Open-Flamingo, Language Model=LLAMA-7B, Shots=8-shot2023.12 | 50.84 | — | |
| SILVLM=Open-Flamingo, Language Model=MPT-7B, Shots=4-shot2023.12 | 50.36 | — | |
| SQLVLM=Open-Flamingo, Language Model=MPT-7B, Shots=8-shot2023.12 | 50.16 | — | |
| SILVLM=Open-Flamingo, Language Model=LLAMA-7B, Shots=8-shot2023.12 | 49.65 | — | |
| SQLVLM=Open-Flamingo, Language Model=MPT-7B, Shots=16-shot2023.12 | 49.16 | — | |
| RSLVLM=Open-Flamingo, Language Model=MPT-7B, Shots=4-shot2023.12 | 48.82 | — | |
| SQLVLM=Open-Flamingo, Language Model=LLAMA-7B, Shots=4-shot2023.12 | 48.82 | — | |
| RSLVLM=Open-Flamingo, Language Model=LLAMA-7B, Shots=16-shot2023.12 | 48.71 | — | |
| SQLVLM=Open-Flamingo, Language Model=MPT-7B, Shots=4-shot2023.12 | 47.49 | — | |
| RSLVLM=Open-Flamingo, Language Model=LLAMA-7B, Shots=8-shot2023.12 | 47.38 | — | |
| SILVLM=Open-Flamingo, Language Model=LLAMA-7B, Shots=4-shot2023.12 | 47.3 | — | |
| RSLVLM=Open-Flamingo, Language Model=LLAMA-7B, Shots=4-shot2023.12 | 44.56 | — | |
| RSLVLM=Otter, Language Model=LLAMA-7B, Shots=16-shot2023.12 | 42.43 | — | |
| SILVLM=Otter, Language Model=LLAMA-7B, Shots=16-shot2023.12 | 41.42 | — | |
| RSLVLM=Otter, Language Model=LLAMA-7B, Shots=8-shot2023.12 | 41.28 | — | |
| SQLVLM=Otter, Language Model=LLAMA-7B, Shots=16-shot2023.12 | 40.57 | — | |
| SILVLM=Otter, Language Model=LLAMA-7B, Shots=8-shot2023.12 | 39.72 | — | |
| SQLVLM=Otter, Language Model=LLAMA-7B, Shots=8-shot2023.12 | 39.41 | — | |
| SQLVLM=Otter, Language Model=LLAMA-7B, Shots=4-shot2023.12 | 39.16 | — | |
| RSLVLM=Otter, Language Model=LLAMA-7B, Shots=4-shot2023.12 | 39.14 | — | |
| SILVLM=Otter, Language Model=LLAMA-7B, Shots=4-shot2023.12 | 37.61 | — | |
| RSLVLM=Otter, Language Model=MPT-7B, Shots=16-shot2023.12 | 29.92 | — | |
| RSLVLM=Otter, Language Model=MPT-7B, Shots=8-shot2023.12 | 27.6 | — | |
| RSLVLM=Otter, Language Model=MPT-7B, Shots=4-shot2023.12 | 24.96 | — | |
| SILVLM=Otter, Language Model=MPT-7B, Shots=8-shot2023.12 | 24.38 | — | |
| SILVLM=Otter, Language Model=MPT-7B, Shots=16-shot2023.12 | 23.95 | — | |
| SQLVLM=Otter, Language Model=MPT-7B, Shots=8-shot2023.12 | 23.56 | — | |
| SQLVLM=Otter, Language Model=MPT-7B, Shots=4-shot2023.12 | 23.29 | — | |
| SILVLM=Otter, Language Model=MPT-7B, Shots=4-shot2023.12 | 23.08 | — | |
| SQLVLM=Otter, Language Model=MPT-7B, Shots=16-shot2023.12 | 22.74 | — | |
| RSLVLM=Open-Flamingo, Language Model=LLAMA-7B2023.12 | — | 46.88 | |
| RSLVLM=Open-Flamingo, Language Model=MPT-7B2023.12 | — | 50.25 | |
| RSLVLM=Otter, Language Model=LLAMA-7B2023.12 | — | 41.13 | |
| RSLVLM=Otter, Language Model=MPT-7B2023.12 | — | 27.49 | |
| SILVLM=Open-Flamingo, Language Model=LLAMA-7B2023.12 | — | 49.55 | |
| SILVLM=Open-Flamingo, Language Model=MPT-7B2023.12 | — | 52.47 | |
| SILVLM=Otter, Language Model=LLAMA-7B2023.12 | — | 39.58 | |
| SILVLM=Otter, Language Model=MPT-7B2023.12 | — | 23.8 | |
| SQLVLM=Open-Flamingo, Language Model=LLAMA-7B2023.12 | — | 50.82 | |
| SQLVLM=Open-Flamingo, Language Model=MPT-7B2023.12 | — | 48.94 | |
| SQLVLM=Otter, Language Model=LLAMA-7B2023.12 | — | 39.71 | |
| SQLVLM=Otter, Language Model=MPT-7B2023.12 | — | 23.2 |