Sequential Visual Reasoning on VisualFLIP
80.8Cardinality AccpGemini 3.5 Flash
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Gemini 3.5 FlashProtocol=Sequential (two-turn), Token Budget=40962026.06 | 80.8 | 8.1 | 66.3 | 22.8 | 65.3 | 9 | 66.1 | 6.8 | 69.1 | 14.2 | |
| Gemini 3.1 ProProtocol=Sequential (two-turn), Token Budget=40962026.06 | 76.7 | 10.1 | 83.2 | 10.5 | 70.7 | 14.5 | 68.6 | 11.3 | 76.6 | 11.4 | |
| Qwen3.6-PlusProtocol=Sequential (two-turn), Token Budget=40962026.06 | 43.2 | 51.1 | 53.1 | 41.3 | 49.3 | 32.5 | 48.3 | 22.1 | 49.3 | 39.2 | |
| Claude Opus 4.7Protocol=Sequential (two-turn), Token Budget=40962026.06 | 34.9 | 22.5 | 64.5 | 23.4 | 50 | 18.6 | 47.5 | 19.8 | 52.1 | 21.6 | |
| GLM-5V-TurboProtocol=Sequential (two-turn), Token Budget=40962026.06 | 18.7 | 79.4 | 19.1 | 38.3 | 28 | 41.9 | 21 | 33.3 | 21.3 | 47.1 | |
| GPT-4oProtocol=Sequential (two-turn), Token Budget=40962026.06 | 16.4 | 25 | 38.1 | 8.5 | 45.3 | 11.6 | 14.4 | 15.6 | 31 | 13.2 | |
| Grok 4.3Protocol=Sequential (two-turn), Token Budget=40962026.06 | 15.1 | 41.4 | 26.7 | 43.8 | 40.7 | 27.1 | 26.3 | 25 | 27.2 | 36.5 | |
| GLM-4.6VProtocol=Sequential (two-turn), Token Budget=40962026.06 | 14.5 | 57.1 | 13.8 | 63.4 | 21.3 | 55.1 | 10.4 | 47.1 | 15 | 57.5 | |
| Qwen3-VL-235BProtocol=Sequential (two-turn), Token Budget=40962026.06 | 11.6 | 72.9 | 13.6 | 68.3 | 36 | 39.4 | 24.6 | 38.5 | 19.9 | 56.7 | |
| GPT-5-miniProtocol=Sequential (two-turn), Token Budget=40962026.06 | 11.6 | 64.8 | 31.9 | 47.4 | 40.7 | 37.1 | 21.2 | 51.5 | 27.7 | 48.4 |