Visual Geometric Reasoning on Eyeballing Tasks
88Ray Intersection AccuracySora-2 (Major Frame)
Evaluation results
| Method | Links | ||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Sora-2 (Major Frame)Input Modality=Video, Frame Selection Strategy=Major Frame, Evaluation Strategy=Manual2025.11 | 88 | — | 64 | 70 | 22 | 44 | 34 | 24 | 26 | 30 | 44.7 | 46 | 30 | 38 | 68 | 40 | 26 | 24 | 32 | 38 | 54 | 32 | 16 | 36 | 34.5 | 40.2 | |
| Sora-2 (Last Frame)Input Modality=Video, Frame Selection Strategy=Last Frame, Evaluation Strategy=Manual2025.11 | 70 | — | 48 | 56 | 22 | 42 | 30 | 20 | 18 | 24 | 36.7 | 38 | 28 | 36 | 56 | 20 | 20 | 16 | 30 | 30.5 | 44 | 28 | 14 | 42 | 32 | 33.4 | |
| Sora-2 (Audio)Input Modality=Audio, Evaluation Strategy=Manual2025.11 | 22 | — | 22 | 58 | 18 | 34 | 48 | 14 | 32 | 24 | 30.2 | 20 | 22 | 28 | 12 | 22 | 22 | 18 | 28 | 21.5 | 54 | 24 | 30 | 36 | 36 | 28 | |
| Gemini 2.5 ProEvaluation Strategy=Rule-based2025.11 | 22 | — | 28 | 44 | 30 | 38 | 32 | 12 | 14 | 30 | 27.8 | 8 | 20 | 28 | 20 | 16 | 22 | 22 | 32 | 21 | 52 | 24 | 38 | 24 | 34.5 | 26.5 | |
| Claude Sonnet 4.5Evaluation Strategy=Rule-based2025.11 | 22 | — | 66 | 50 | 30 | 36 | 34 | 26 | 28 | 34 | 36.2 | 14 | 32 | 24 | 12 | 58 | 22 | 22 | 26 | 26.3 | 86 | 36 | 60 | 20 | 50.5 | 35.1 | |
| GPT-5 highEvaluation Strategy=Rule-based2025.11 | 16 | — | 34 | 62 | 28 | 40 | 30 | 32 | 32 | 28 | 33.6 | 26 | 32 | 28 | 20 | 30 | 20 | 18 | 18 | 24 | 54 | 30 | 20 | 26 | 32.5 | 29.7 |