Omnidirectional Image Understanding on ODI-Bench open-ended 1.0 (test)
49.53Overall Scoreo3
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| o3Model Category=Proprietary MLLMs, Evaluation Setting=open-ended2025.10 | 49.53 | 55.49 | 45.36 | 69.5 | 50 | 52.89 | 45.4 | 34.59 | 39.6 | 62.04 | 59.1 | |
| GPT-4oModel Category=Proprietary MLLMs, Evaluation Setting=open-ended2025.10 | 42.91 | 52.62 | 39.74 | 68.5 | 45.35 | 43.8 | 32.27 | 27.25 | 30.3 | 61.84 | 49.5 | |
| InternVL3-78BModel Category=Open-sourced MLLMs, Evaluation Setting=open-ended2025.10 | 42.52 | 47.05 | 43.91 | 62 | 57.56 | 48.76 | 35.58 | 29.77 | 29.82 | 53.27 | 53.95 | |
| Intern-s1Model Category=Open-sourced MLLMs, Evaluation Setting=open-ended2025.10 | 42.12 | 46.3 | 41.68 | 53.5 | 63.95 | 40.5 | 34.72 | 31.87 | 30.8 | 52.04 | 58.85 | |
| InternVL3-38BModel Category=Open-sourced MLLMs, Evaluation Setting=open-ended2025.10 | 40.96 | 48.11 | 39.57 | 59 | 40.94 | 38.43 | 34.66 | 29.77 | 27.6 | 54.08 | 52.62 | |
| Qwen-VL-PlusModel Category=Proprietary MLLMs, Evaluation Setting=open-ended2025.10 | 39.87 | 44.39 | 39.35 | 67.5 | 47 | 36.36 | 34.39 | 29.1 | 27.65 | 51.09 | 46.2 | |
| Qwen2.5-VL-72BModel Category=Open-sourced MLLMs, Evaluation Setting=open-ended2025.10 | 39.49 | 45.89 | 35.14 | 63 | 42.44 | 39.55 | 34.91 | 24.1 | 27.6 | 54.39 | 47.75 | |
| Qwen2.5-VL-3BModel Category=Open-sourced MLLMs, Evaluation Setting=open-ended2025.10 | 38.91 | 40.8 | 39.64 | 63.5 | 41.86 | 41.32 | 35.77 | 28.09 | 26.8 | 47.35 | 56.2 | |
| Qwen2.5-VL-32BModel Category=Open-sourced MLLMs, Evaluation Setting=open-ended2025.10 | 37.67 | 39.56 | 26.34 | 62 | 42.44 | 30.91 | 36.47 | 30.21 | 30.43 | 50.41 | 44.25 | |
| Gemini-2.0-flashModel Category=Proprietary MLLMs, Evaluation Setting=open-ended2025.10 | 36.42 | 37.82 | 28.55 | 48.26 | 50 | 56.5 | 30.86 | 25.89 | 31 | 55.51 | 42.17 | |
| glm-4v-9BModel Category=Open-sourced MLLMs, Evaluation Setting=open-ended2025.10 | 35.79 | 38.8 | 26.12 | 60.5 | 42.44 | 32.23 | 36.38 | 25.26 | 26.5 | 41.43 | 42.95 | |
| LLava-Next-34BModel Category=Open-sourced MLLMs, Evaluation Setting=open-ended2025.10 | 35.25 | 41.57 | 29.28 | 49.5 | 41.52 | 7.025 | 35.4 | 20.21 | 24.6 | 45.51 | 52.42 | |
| MiniCPM-V 4.0Model Category=Open-sourced MLLMs, Evaluation Setting=open-ended2025.10 | 32.52 | 36.02 | 30.56 | 57 | 43.6 | 38.43 | 29.34 | 24.32 | 21.04 | 46.53 | 36.25 | |
| LLaVA-v1.5-7BModel Category=Open-sourced MLLMs, Evaluation Setting=open-ended2025.10 | 32.29 | 35.02 | 32.07 | 56.5 | 24.42 | 9.504 | 28.28 | 25.47 | 26.6 | 45.1 | 43.5 | |
| InternVL2.5-8BModel Category=Open-sourced MLLMs, Evaluation Setting=open-ended2025.10 | 30.86 | 33.52 | 22.34 | 62 | 39.53 | 34.71 | 21.96 | 25.68 | 20.4 | 38.98 | 51.55 | |
| LLava-Next-7BModel Category=Open-sourced MLLMs, Evaluation Setting=open-ended2025.10 | 30.52 | 36.58 | 32.24 | 60.5 | 40.12 | 12.81 | 21.04 | 22.01 | 17.6 | 38.78 | 44.45 | |
| XComposer2Model Category=Open-sourced MLLMs, Evaluation Setting=open-ended2025.10 | 28.36 | 30.12 | 20.92 | 48 | 34.88 | 1.652 | 23.13 | 25.79 | 23 | 42.65 | 43.15 | |
| idefics3-8BModel Category=Open-sourced MLLMs, Evaluation Setting=open-ended2025.10 | 27.93 | 31.5 | 28.16 | 65 | 43.6 | 27.27 | 16.75 | 15.41 | 20.6 | 37.35 | 37.9 | |
| Deepseek-VL-1.3BModel Category=Open-sourced MLLMs, Evaluation Setting=open-ended2025.10 | 27.8 | 31.33 | 18.22 | 53.5 | 33.14 | 6.198 | 20.98 | 19.92 | 23.1 | 42.24 | 44.15 | |
| LLaVA-ov-0.5BModel Category=Open-sourced MLLMs, Evaluation Setting=open-ended2025.10 | 17.9 | 28.75 | 11.12 | 42 | 19.19 | 3.719 | 6.196 | 5.765 | 8.3 | 29.18 | 32.2 |