Visual Question Answering on GQA (Accuracy, Cost)
75.77AccuracyOriginal
Evaluation results
| Method | Links | ||
|---|---|---|---|
| OriginalModel=LLaVA-1.5-7B-hf2026.02 | 75.77 | 120 | |
| OriginalModel=Qwen2-VL-72B-Instruct2026.02 | 74.35 | 120 | |
| JPEG Q10Model=LLaVA-1.5-7B-hf2026.02 | 74.11 | 63.9 | |
| OriginalModel=Pixtral-12B-24092026.02 | 73.52 | 120 | |
| OriginalModel=Qwen3-VL-235B-A22B-Instruct2026.02 | 72.98 | 120 | |
| JPEG Q10Model=Qwen2-VL-72B-Instruct2026.02 | 71.57 | 63.9 | |
| VOIModel=LLaVA-1.5-7B-hf2026.02 | 71.18 | 26.86 | |
| VOIModel=Qwen2-VL-72B-Instruct2026.02 | 71.03 | 45.37 | |
| JPEG Q10Model=Qwen3-VL-235B-A22B-Instruct2026.02 | 70.4 | 63.9 | |
| JPEG Q10Model=Pixtral-12B-24092026.02 | 69.81 | 63.9 | |
| VOIModel=Qwen3-VL-235B-A22B-Instruct2026.02 | 68.28 | 44.7 | |
| VOIModel=Pixtral-12B-24092026.02 | 67.37 | 45.66 | |
| Resize 32x32Model=LLaVA-1.5-7B-hf2026.02 | 67.37 | 18.1 | |
| OriginalModel=Qwen2.5-VL-7B-Instruct2026.02 | 67.22 | 120 | |
| CaptionModel=LLaVA-1.5-7B-hf2026.02 | 64.09 | 9.1 | |
| OriginalModel=Llama-4-Maverick (17Bx12E)2026.02 | 63.56 | 120 | |
| JPEG Q1Model=LLaVA-1.5-7B-hf2026.02 | 63.51 | 45.4 | |
| JPEG Q10Model=Llama-4-Maverick (17Bx12E)2026.02 | 63.17 | 63.9 | |
| VOIModel=Qwen2.5-VL-7B-Instruct2026.02 | 62.92 | 59.66 | |
| JPEG Q1Model=Qwen2-VL-72B-Instruct2026.02 | 62.68 | 45.4 | |
| JPEG Q1Model=Qwen3-VL-235B-A22B-Instruct2026.02 | 62.58 | 45.4 | |
| VOIModel=Llama-4-Maverick (17Bx12E)2026.02 | 60.87 | 41.64 | |
| JPEG Q1Model=Pixtral-12B-24092026.02 | 60.53 | 45.4 | |
| JPEG Q10Model=Qwen2.5-VL-7B-Instruct2026.02 | 60.14 | 63.9 | |
| Resize 32x32Model=Qwen3-VL-235B-A22B-Instruct2026.02 | 54.08 | 18.1 | |
| JPEG Q1Model=Llama-4-Maverick (17Bx12E)2026.02 | 52.76 | 45.4 | |
| Resize 32x32Model=Qwen2-VL-72B-Instruct2026.02 | 49.63 | 18.1 | |
| JPEG Q1Model=Qwen2.5-VL-7B-Instruct2026.02 | 48.66 | 45.4 | |
| Resize 32x32Model=Qwen2.5-VL-7B-Instruct2026.02 | 46.7 | 18.1 | |
| Resize 32x32Model=Llama-4-Maverick (17Bx12E)2026.02 | 43.92 | 18.1 | |
| CaptionModel=Pixtral-12B-24092026.02 | 42.45 | 9.1 | |
| CaptionModel=Qwen2.5-VL-7B-Instruct2026.02 | 41.28 | 9.1 | |
| Resize 32x32Model=Pixtral-12B-24092026.02 | 38.1 | 18.1 | |
| CaptionModel=Llama-4-Maverick (17Bx12E)2026.02 | 34.93 | 9.1 | |
| CaptionModel=Qwen2-VL-72B-Instruct2026.02 | 32.34 | 9.1 | |
| CaptionModel=Qwen3-VL-235B-A22B-Instruct2026.02 | 28.97 | 9.1 |