Visual Question Answering on IRIS 1.0 (ambiguous trials)
59.3Accuracy (Image Only)Gemini 2.5 Pro
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Gemini 2.5 ProTemporal Window=±1s around speech onset2026.02 | 59.3 | 82.3 | 23 | — | |
| Qwen 2.5 VL 32BTemporal Window=±1s around speech onset2026.02 | 56.2 | 73 | 16.7 | — | |
| Claude 4.1 OpusTemporal Window=±1s around speech onset2026.02 | 55.2 | 72 | 16.7 | — | |
| Gemini 2.5 FlashTemporal Window=±1s around speech onset2026.02 | 54.2 | 83.3 | 29 | — | |
| Claude Sonnet 4Temporal Window=±1s around speech onset2026.02 | 54.2 | 74 | 19.8 | — | |
| GPT-5Temporal Window=±1s around speech onset2026.02 | 53.7 | 76.5 | 22.8 | — | |
| Ovis 2.5 9BTemporal Window=±1s around speech onset2026.02 | 52.5 | 69.5 | 17 | — | |
| Qwen 2.5 VL 3BTemporal Window=±1s around speech onset2026.02 | 52 | 54.5 | 2.5 | 0.48 | |
| Qwen 2.5 VL 7BTemporal Window=±1s around speech onset2026.02 | 51.7 | 63.7 | 12 | — | |
| GPT-5 MiniTemporal Window=±1s around speech onset2026.02 | 49.7 | 83 | 33.2 | — |