Visual Question Answering on VQA v2 (Accuracy and Cost)
79.01AccuracyOriginal
Evaluation results
| Method | Links | ||
|---|---|---|---|
| OriginalModel=Qwen3-VL-235B-A22B-Instruct2026.02 | 79.01 | 120 | |
| OriginalModel=Qwen2.5-VL-7B-Instruct2026.02 | 77.98 | 120 | |
| OriginalModel=Qwen2-VL-72B-Instruct2026.02 | 77.16 | 120 | |
| VOIModel=Qwen3-VL-235B-A22B-Instruct2026.02 | 74.57 | 52.04 | |
| JPEG Q10Model=Qwen2-VL-72B-Instruct2026.02 | 74.27 | 63.9 | |
| OriginalModel=Pixtral-12B-24092026.02 | 74.06 | 120 | |
| VOIModel=Qwen2-VL-72B-Instruct2026.02 | 73.01 | 48 | |
| OriginalModel=LLaVA-1.5-7B-hf2026.02 | 72.8 | 120 | |
| JPEG Q10Model=Qwen3-VL-235B-A22B-Instruct2026.02 | 72.58 | 63.9 | |
| JPEG Q10Model=Qwen2.5-VL-7B-Instruct2026.02 | 72.1 | 63.9 | |
| VOIModel=Qwen2.5-VL-7B-Instruct2026.02 | 72.01 | 57.31 | |
| JPEG Q10Model=LLaVA-1.5-7B-hf2026.02 | 70.21 | 63.9 | |
| JPEG Q10Model=Pixtral-12B-24092026.02 | 69.21 | 63.9 | |
| VOIModel=Pixtral-12B-24092026.02 | 68.67 | 45.41 | |
| VOIModel=LLaVA-1.5-7B-hf2026.02 | 68.07 | 38.52 | |
| OriginalModel=Llama-4-Maverick (17Bx12E)2026.02 | 67.37 | 120 | |
| JPEG Q10Model=Llama-4-Maverick (17Bx12E)2026.02 | 65.99 | 63.9 | |
| VOIModel=Llama-4-Maverick (17Bx12E)2026.02 | 64.61 | 43.37 | |
| JPEG Q1Model=Qwen3-VL-235B-A22B-Instruct2026.02 | 62.62 | 45.4 | |
| JPEG Q1Model=Qwen2-VL-72B-Instruct2026.02 | 60.49 | 45.4 | |
| Resize 32x32Model=LLaVA-1.5-7B-hf2026.02 | 58.99 | 18.1 | |
| JPEG Q1Model=LLaVA-1.5-7B-hf2026.02 | 58.25 | 45.4 | |
| CaptionModel=LLaVA-1.5-7B-hf2026.02 | 58.01 | 9.1 | |
| CaptionModel=Qwen3-VL-235B-A22B-Instruct2026.02 | 57.36 | 9.1 | |
| JPEG Q1Model=Pixtral-12B-24092026.02 | 57.27 | 45.4 | |
| JPEG Q1Model=Qwen2.5-VL-7B-Instruct2026.02 | 56.26 | 45.4 | |
| JPEG Q1Model=Llama-4-Maverick (17Bx12E)2026.02 | 50.45 | 45.4 | |
| Resize 32x32Model=Qwen3-VL-235B-A22B-Instruct2026.02 | 50.21 | 18.1 | |
| CaptionModel=Pixtral-12B-24092026.02 | 49.83 | 9.1 | |
| Resize 32x32Model=Qwen2.5-VL-7B-Instruct2026.02 | 48.38 | 18.1 | |
| CaptionModel=Llama-4-Maverick (17Bx12E)2026.02 | 45.98 | 9.1 | |
| Resize 32x32Model=Qwen2-VL-72B-Instruct2026.02 | 43.9 | 18.1 | |
| CaptionModel=Qwen2-VL-72B-Instruct2026.02 | 40.21 | 9.1 | |
| Resize 32x32Model=Llama-4-Maverick (17Bx12E)2026.02 | 39.42 | 18.1 | |
| CaptionModel=Qwen2.5-VL-7B-Instruct2026.02 | 36.65 | 9.1 | |
| Resize 32x32Model=Pixtral-12B-24092026.02 | 33.8 | 18.1 |