Visual Question Answering on CVQA (Global, SEA, Others)
67.59Global Accuracy (CVQA)SEA-Gemma-3 10%
Evaluation results
| Method | Links | |||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SEA-Gemma-3 10%Number of Parameters=27B2026.04 | 67.59 | 69.12 | 67.06 | 66.26 | 70.94 | 71.43 | 75.14 | 74.3 | 63.64 | 67.73 | 63.5 | 62.39 | 75.87 | 45.68 | 64.42 | 65.02 | 81.09 | 47 | 65.95 | 53.69 | 53.19 | 77.93 | 77.23 | 50.64 | 69.57 | 47.2 | 80.63 | 76.49 | 81.5 | 65.78 | 69.92 | 80.95 | 74.5 | 75.69 | — | |
| Gemma-3Number of Parameters=27B2026.04 | 65.69 | 66.58 | 65.39 | 66.5 | 66.01 | 67.62 | 72.66 | 73.36 | 62.96 | 64.54 | 59 | 62.82 | 72.03 | 42.96 | 62.26 | 62.07 | 80.6 | 43.5 | 64.11 | 53.2 | 53.62 | 76.9 | 74.75 | 50.32 | 66.89 | 45.79 | 80.63 | 75.17 | 80.5 | 65.33 | 69.34 | 76.19 | 71.5 | 73.39 | — | |
| MolmoNumber of Parameters=72B2026.04 | 60.86 | 62.62 | 60.25 | 63.83 | 64.53 | 69.84 | 79.29 | 58.41 | 54.88 | 58.17 | 52 | 45.3 | 68.88 | 35.06 | 57.68 | 58.62 | 78.11 | 41.5 | 57.06 | 57.14 | 40.43 | 74.14 | 68.81 | 47.76 | 69.9 | 42.06 | 77.46 | 70.2 | 84 | 45.78 | 71.55 | 67.77 | 57 | 69.5 | — | |
| Aya-VisionNumber of Parameters=32B2026.04 | 60.18 | 64.45 | 58.7 | 67.88 | 66.34 | 72.38 | 77.48 | 61.68 | 55.56 | 61.75 | 52.53 | 29.18 | 64.31 | 39.36 | 56.49 | 68.47 | 78.11 | 38 | 56.13 | 59.11 | 40.43 | 80 | 66.17 | 36.01 | 66.22 | 36.45 | 78.01 | 74.17 | 80 | 39.56 | 73.25 | 66.18 | 57.79 | 66.69 | — | |
| Qwen-2.5-VLNumber of Parameters=72B2026.04 | 59.08 | 62.17 | 58.01 | 66.5 | 65.02 | 62.5 | 80.98 | 58.88 | 55.22 | 51.79 | 56.5 | 36.48 | 61.27 | 37.78 | 61.99 | 61.08 | 75.12 | 36.55 | 57.98 | 58.62 | 38.3 | 77.59 | 61.39 | 39.1 | 68.56 | 35.05 | 76.76 | 75.83 | 79 | 45.5 | 71.44 | 55.31 | 58.5 | 64.92 | — | |
| Qwen-3 VLNumber of Parameters=35B2026.04 | 58.83 | 63.18 | 57.32 | 66.26 | 64.04 | 70.16 | 78.28 | 61.68 | 54.55 | 54.98 | 55.5 | 42.74 | 64.34 | 34.81 | 54.45 | 57.64 | 75.62 | 34.5 | 55.52 | 58.62 | 32.77 | 77.24 | 63.86 | 44.23 | 61.2 | 38.32 | 76.06 | 70.2 | 76.5 | 43.56 | 68.2 | 60.07 | 61 | 66.99 | — | |
| Llama-3.2 VisionNumber of Parameters=11B2026.04 | 51.53 | 53.63 | 50.81 | 56.31 | 51.72 | 56.19 | 62.8 | 58.41 | 47.81 | 51.79 | 44 | 39.32 | 55.59 | 34.57 | 49.06 | 49.26 | 68.16 | 44 | 53.99 | 50.74 | 35.32 | 59.66 | 48.02 | 39.42 | 54.52 | 34.11 | 57.75 | 58.94 | 66.5 | 48 | 57.02 | 53.11 | 55.5 | 55.98 | — | |
| PixtralNumber of Parameters=12B2026.04 | 49.01 | 57.71 | 45.99 | 62.86 | 64.53 | 61.9 | 68.93 | 51.87 | 51.18 | 51.39 | 49 | 32.91 | 48.25 | 35.8 | 22.91 | 43.35 | 30.85 | 41.5 | 57.67 | 49.26 | 34.47 | 73.45 | 31.19 | 39.74 | 64.21 | 35.51 | 73.59 | 67.88 | 37 | 28.44 | 69.13 | 60.07 | 32.5 | 48.02 | — | |
| SEA-Gemma-3Number of Parameters=27B2026.04 | 31.94 | 26.95 | 33.68 | 21.6 | 24.14 | 29.84 | 40.73 | 40.65 | 22.22 | 23.9 | 12.5 | 26.07 | 33.57 | 27.65 | 34.23 | 32.02 | 42.79 | 25.5 | 31.29 | 29.06 | 28.51 | 42.76 | 34.16 | 23.4 | 38.8 | 24.77 | 41.2 | 40.73 | 45.5 | 29.78 | 37.61 | 36.26 | 32 | 36.93 | — | |
| SEA-SigLIP2Merge weight (beta)=50%, Merge strategy=linear merge, Evaluation protocol=Zero-Shot2026.04 | 27.52 | 25.5 | 27.97 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SEA-SigLIP2Merge weight (beta)=75%, Merge strategy=linear merge, Evaluation protocol=Zero-Shot2026.04 | 27.12 | 27.51 | 27.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SEA-SigLIP2 (w/o merge)Merge strategy=w/o merge, Evaluation protocol=Zero-Shot2026.04 | 26.75 | 25.29 | 27.07 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Google SigLIP2Evaluation protocol=Zero-Shot2026.04 | 25.51 | 24.02 | 25.84 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SEA-SigLIP2Merge weight (beta)=25%, Merge strategy=linear merge, Evaluation protocol=Zero-Shot2026.04 | 24.38 | 24.44 | 24.36 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BreadcrumbsModel=Breadcrumbs2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 34.24 | |
| DAREModel=DARE2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 38.42 | |
| DiM3-7BModel=DiM3-7B2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 43.78 | |
| LLaVA-v1.5-7BModel=LLaVA-v1.5-7B2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 39.32 | |
| NeuroMergingModel=NeuroMerging2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 39.38 | |
| Palo-13BModel=Palo-13B2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 39.59 | |
| Palo-7BModel=Palo-7B2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 37 | |
| PCB-MergingModel=PCB-Merging2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 7.8 | |
| STFModel=STF2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 34.59 | |
| Task ArithmeticModel=Task Arithmetic2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 40.47 | |
| TIES-MergingModel=TIES-Merging2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 39.22 |