Omnidirectional Image Understanding on ODI-Bench close-ended 1.0
62.62Overall Scoreo3
Evaluation Results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| o3evaluation_setting=close-ended2025.10 | 62.62 | 75.82 | 74.01 | 75 | 57.56 | 77.69 | 56.2 | 39.62 | 46.6 | 70.2 | 59.5 | |
| InternVL3-78Bevaluation_setting=close-ended2025.10 | 59.43 | 79.18 | 77.3 | 66.5 | 59.3 | 80.99 | 46.01 | 31.67 | 40.4 | 60.82 | 58.5 | |
| InternVL3-38Bevaluation_setting=close-ended2025.10 | 57.91 | 75.57 | 76.32 | 69 | 54.07 | 76.86 | 46.56 | 32.29 | 40.4 | 55.92 | 56.5 | |
| Gemini-2.0-flashevaluation_setting=close-ended2025.10 | 57.12 | 73.03 | 69.41 | 66.5 | 52.33 | 80.16 | 48.1 | 32.91 | 40.2 | 61.13 | 54 | |
| Qwen2.5-VL-72Bevaluation_setting=close-ended2025.10 | 56.91 | 77.38 | 68.75 | 52 | 58.14 | 75.21 | 46.75 | 32.08 | 38.4 | 60 | 50 | |
| Qwen2.5-VL-32Bevaluation_setting=close-ended2025.10 | 56.7 | 74.69 | 67.76 | 60 | 57.56 | 72.73 | 45.97 | 35.01 | 38.2 | 59.59 | 54.5 | |
| GPT-4oevaluation_setting=close-ended2025.10 | 55.79 | 74.43 | 67.76 | 65.5 | 49.42 | 74.38 | 43.24 | 32.49 | 39.6 | 57.55 | 53.5 | |
| Qwen-VL-Plusevaluation_setting=close-ended2025.10 | 53.85 | 74.51 | 68.75 | 43 | 52.33 | 67.77 | 46.13 | 29.98 | 34.8 | 50.61 | 49 | |
| MiniCPM-V 4.0evaluation_setting=close-ended2025.10 | 53.71 | 71.72 | 68.42 | 67 | 51.74 | 74.38 | 42.58 | 32.29 | 33 | 49.39 | 51 | |
| glm-4v-9Bevaluation_setting=close-ended2025.10 | 53.2 | 70.41 | 64.14 | 69 | 54.65 | 69.42 | 44.29 | 32.91 | 30.8 | 43.27 | 57.5 | |
| Qwen2.5-VL-3Bevaluation_setting=close-ended2025.10 | 52.88 | 70.66 | 65.13 | 64 | 45.93 | 71.07 | 46.13 | 28.6 | 33 | 45.71 | 53.5 | |
| InternVL2.5-8Bevaluation_setting=close-ended2025.10 | 52.76 | 68.52 | 70.07 | 60 | 51.74 | 66.12 | 45.23 | 31.24 | 33 | 44.08 | 58 | |
| LLava-Next-34Bevaluation_setting=close-ended2025.10 | 52.24 | 70.57 | 62.17 | 54.5 | 45.93 | 45.45 | 47.42 | 26.42 | 38 | 45.71 | 57.5 | |
| XComposer2evaluation_setting=close-ended2025.10 | 51.84 | 75.57 | 76.32 | 44 | 54.07 | 25.62 | 46.56 | 32.29 | 27.2 | 31.84 | 46 | |
| idefics3-8Bevaluation_setting=close-ended2025.10 | 49.89 | 68.45 | 65.32 | 62.5 | 50 | 57.85 | 36.71 | 28.79 | 30.2 | 49.39 | 49.5 | |
| LLava-Next-7Bevaluation_setting=close-ended2025.10 | 45.91 | 64.84 | 54.93 | 60.5 | 39.77 | 45.45 | 40.42 | 27.04 | 21.2 | 32.24 | 53.5 | |
| LLaVA-v1.5-7Bevaluation_setting=close-ended2025.10 | 45.04 | 64.02 | 51.44 | 58 | 34.3 | 42.15 | 43.86 | 28.93 | 19.6 | 24.08 | 50 | |
| LLaVA-ov-0.5Bevaluation_setting=close-ended2025.10 | 44.25 | 61.21 | 61.26 | 42 | 44.77 | 27.27 | 30.61 | 29.77 | 35.4 | 50.2 | 32 | |
| Intern-s1evaluation_setting=close-ended2025.10 | 42.37 | 63.93 | 51.97 | 42.5 | 40.94 | 33.06 | 20.15 | 32.08 | 31 | 45.31 | 43 | |
| Deepseek-VL-1.3Bevaluation_setting=close-ended2025.10 | 42.02 | 55.66 | 50 | 52.5 | 39.53 | 29.75 | 39.02 | 28.18 | 28 | 23.27 | 49 | |
| Blind GPT-4oevaluation_setting=close-ended, input=no image2025.10 | 36.39 | 58.93 | 42.43 | 17 | 17.44 | 29.75 | 21.96 | 29.14 | 31.6 | 29.8 | 25.5 | |
| Random Choiceevaluation_setting=close-ended2025.10 | 26.93 | 25 | 25 | 50 | 25 | 25 | 25 | 25 | 25 | 25 | 41 |