Visual Question Answering on VizWiz-QA (val)
88.7AUCLikelihood
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| LikelihoodModel=Flamingo 3B, Shots=4-shot, Input=Image2024.06 | 88.7 | 17.3 | 22.1 | 12.1 | 3.5 | |
| AVG BLEUModel=Flamingo 3B, Shots=4-shot, Input=Image2024.06 | 88.1 | 19.5 | 18.1 | 9.5 | 4.7 | |
| DiversityModel=Falcon, Shots=4-shot, Input=Gold image captions2024.06 | 86.2 | 31.3 | 11.8 | 0 | 0 | |
| AVG BLEUModel=Falcon, Shots=4-shot, Input=Gold image captions2024.06 | 85.1 | 13.5 | 12.1 | 4.6 | 1.1 | |
| RepetitionsModel=Falcon, Shots=4-shot, Input=Gold image captions2024.06 | 84 | 34 | 11.8 | 0 | 0 | |
| DiversityModel=Flamingo 3B, Shots=4-shot, Input=Image2024.06 | 77.6 | 12.4 | 11.4 | 7.2 | 3.7 | |
| RepetitionsModel=Flamingo 3B, Shots=4-shot, Input=Image2024.06 | 76.7 | 5.4 | 7.9 | 7.9 | 4.4 | |
| DiversityModel=PaLM 2 Bison, Shots=4-shot, Input=Gold image captions2024.06 | 75 | 39.9 | 10.1 | 10.1 | 10.1 | |
| LikelihoodModel=Falcon, Shots=4-shot, Input=Gold image captions2024.06 | 73.8 | 64.2 | 7.2 | 1.7 | 1.4 | |
| AVG BLEUModel=PaLM 2 Bison, Shots=4-shot, Input=Gold image captions2024.06 | 73.6 | 9.8 | 26.4 | 18.4 | 11.2 | |
| RepetitionsModel=PaLM 2 Bison, Shots=4-shot, Input=Gold image captions2024.06 | 72.7 | 31.8 | 21.6 | 10.1 | 10.1 | |
| AVG BLEUModel=LLaVA 13B, Shots=0-shot, Input=Image2024.06 | 70.5 | 28.6 | — | 73.8 | 33.3 | |
| DiversityModel=LLaVA 13B, Shots=0-shot, Input=Image2024.06 | 70.4 | 6.8 | — | 67.2 | 8.7 | |
| RepetitionsModel=LLaVA 13B, Shots=0-shot, Input=Image2024.06 | 69.3 | 9.9 | — | 73.2 | 28.4 | |
| LikelihoodModel=LLaVA 13B, Shots=0-shot, Input=Image2024.06 | 68.8 | 7.9 | — | 71.6 | 20.8 | |
| LikelihoodModel=PaLM 2 Bison, Shots=4-shot, Input=Gold image captions2024.06 | 57.5 | 20.6 | 0 | 0 | 0 |