Visual Reasoning on VisualFLIP
85.6Cardinality AccuracyGPT-5.5
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-5.5Year=2026, Source Category=CLOSED-SOURCE2026.06 | 85.6 | 3.7 | 86.4 | 5.7 | 68 | 7.8 | 65.3 | 6.6 | 78.6 | 5.8 | |
| Gemini 3.5 FlashYear=2026, Source Category=CLOSED-SOURCE2026.06 | 84.9 | 7.1 | 90.5 | 4.5 | 70.7 | 11.4 | 68.6 | 9.5 | 81.2 | 7.3 | |
| Qwen3.6-PlusYear=2026, Source Category=CLOSED-SOURCE2026.06 | 83.6 | 8.3 | 90.5 | 3.4 | 69.3 | 12.8 | 66.1 | 5.3 | 80.2 | 6.8 | |
| Gemini 3.1 ProYear=2026, Source Category=CLOSED-SOURCE2026.06 | 79.5 | 8.6 | 88.3 | 5.9 | 68.7 | 16 | 59.3 | 15.5 | 77.1 | 10.1 | |
| Qwen3.5-FlashYear=2026, Source Category=CLOSED-SOURCE2026.06 | 74 | 6.4 | 87.2 | 3.8 | 52.7 | 17.3 | 59.3 | 7.9 | 72.1 | 7.8 | |
| Seed 2.0 MiniYear=2026, Source Category=CLOSED-SOURCE2026.06 | 61.7 | 10.1 | 86.2 | 8.8 | 67.4 | 18.4 | 40.2 | 14.5 | 68.6 | 11.8 | |
| Kimi K2.6-1TYear=2026, Source Category=OPEN-SOURCE2026.06 | 48.6 | 2.9 | 39.9 | 1.7 | 41.6 | 2.4 | 35.6 | 5.6 | 41.6 | 2.9 | |
| MiMo-v2.5-310BYear=2026, Source Category=OPEN-SOURCE2026.06 | 47.3 | 8.8 | 59 | 7.6 | 46.2 | 15.7 | 43.2 | 8.6 | 51 | 9.6 | |
| Claude Opus 4.7Year=2026, Source Category=CLOSED-SOURCE2026.06 | 43.2 | 16 | 74 | 8.3 | 48.7 | 20.3 | 46.6 | 12.2 | 57.2 | 13.1 | |
| GLM-5V-TurboYear=2026, Source Category=CLOSED-SOURCE2026.06 | 41.1 | 13.3 | 78.7 | 5.5 | 53.6 | 17.3 | 46.6 | 11.1 | 59.7 | 10.2 | |
| GLM-4.6V-106BYear=2025, Source Category=OPEN-SOURCE2026.06 | 37 | 14 | 42.9 | 32.5 | 44.7 | 32.3 | 23.7 | 14.3 | 38.7 | 26.6 | |
| Grok 4.3Year=2026, Source Category=CLOSED-SOURCE2026.06 | 33.6 | 22.9 | 73.6 | 12 | 40 | 28.8 | 43.2 | 17.2 | 52.5 | 18.4 | |
| GPT-5-miniYear=2025, Source Category=CLOSED-SOURCE2026.06 | 28.1 | 27.3 | 53.8 | 29 | 47.3 | 30.4 | 44.1 | 20.2 | 45.3 | 27.6 | |
| Qwen3-VL-32BYear=2025, Source Category=OPEN-SOURCE2026.06 | 27.4 | 50.5 | 16.8 | 67.4 | 42 | 36.6 | 19.5 | 37.5 | 25 | 51.4 | |
| Qwen3-VL-235BYear=2025, Source Category=OPEN-SOURCE2026.06 | 26.7 | 47.2 | 19 | 60.1 | 38.7 | 39.7 | 28 | 24.2 | 26.5 | 47.7 | |
| Claude Opus 4.6Year=2026, Source Category=CLOSED-SOURCE2026.06 | 19.2 | 25 | 38.1 | 17.3 | 30.7 | 29.1 | 14.4 | 32.7 | 28.4 | 23.2 | |
| Qwen3-VL-8BYear=2025, Source Category=OPEN-SOURCE2026.06 | 15.8 | 55.1 | 11.7 | 61.5 | 28 | 43.2 | 20.3 | 48.5 | 17.6 | 52.7 | |
| GPT-4oYear=2024, Source Category=CLOSED-SOURCE2026.06 | 13 | 57.1 | 20.9 | 61.7 | 40 | 34.7 | 20.3 | 34.9 | 23.3 | 50.6 | |
| CoF-7BYear=2025, Source Category=OPEN-SOURCE, TOOL-AUGMENTED2026.06 | 7.5 | 33.3 | 13.2 | 54.7 | 14.7 | 42 | 3.4 | 49 | 10.6 | 47.4 | |
| DeepEyesV2-7BYear=2026, Source Category=OPEN-SOURCE, TOOL-AUGMENTED2026.06 | 7.5 | 34.4 | 8.4 | 50.7 | 11.3 | 41.5 | 10.2 | 32.8 | 9.2 | 42.6 | |
| PixelReasoner-8BYear=2025, Source Category=OPEN-SOURCE, TOOL-AUGMENTED2026.06 | 4.8 | 38.1 | 12.1 | 55.1 | 5.3 | 56.2 | 5.1 | 50 | 7.9 | 51.6 | |
| Mini-o3-7BYear=2025, Source Category=OPEN-SOURCE, TOOL-AUGMENTED2026.06 | 4.8 | 38.3 | 10.6 | 49.6 | 5.3 | 42.1 | 5.1 | 42 | 7.3 | 44.7 | |
| DeepEyes-7BYear=2025, Source Category=OPEN-SOURCE, TOOL-AUGMENTED2026.06 | 4.1 | 42.1 | 8.1 | 64.6 | 8 | 45.6 | 7.6 | 44.3 | 7.1 | 53 | |
| Qwen2.5-VL-7BYear=2025, Source Category=OPEN-SOURCE2026.06 | 3.4 | 45.2 | 12.8 | 61.2 | 6.7 | 53.6 | 11 | 35.2 | 9.2 | 53 |