Visual Reasoning on VisualPuzzles OOD (test)
47.95Overall AccuracyGemini-3.1Pro
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Gemini-3.1Pro2026.05 | 47.95 | 60.69 | 61.14 | 52 | 28.23 | 38.11 | |
| GPT-4o2026.05 | 41.3 | 49.2 | 58.3 | 49 | 27.3 | 26.2 | |
| Monet2026.05 | 35.02 | 45.8 | 30.81 | 47.5 | 26.79 | 25.52 | |
| SCOLAR-7BParameters=7B2026.05 | 34.42 | 33.59 | 29.86 | 49 | 29.67 | 31.82 | |
| Deepeyes2026.05 | 32.96 | 37.79 | 27.01 | 41 | 26.79 | 27.01 | |
| SCOLAR(SFT only)Mode=SFT only2026.05 | 32.88 | 37.79 | 29.38 | 44.5 | 28.71 | 25.87 | |
| Qwen2.5-VLParameters=7B2026.05 | 32.71 | 37.02 | 21.8 | 47.5 | 26.32 | 21.8 | |
| PangeaParameters=7B2026.05 | 31.3 | 32.4 | 23.7 | 38.5 | 28.7 | 32.5 |