Omnidirectional Visual Understanding on 360Bench 1.0 (test)
96FP-IR AccuracyHuman
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| HumanProjection format=—, Inf. Time (seconds)=28.92026.03 | 96 | 82 | 89 | 86 | 78 | 82 | 84 | 90 | 87 | 88 | 86.3 | |
| Gemini Pro 2.5Projection format=ERP, Inf. Time (seconds)=18.42026.03 | 63.2 | 43.5 | 53.3 | 41.1 | 34.7 | 37.9 | 38 | 42.9 | 40.5 | 44 | 43.9 | |
| Gemini Flash 2.5Projection format=ERP, Inf. Time (seconds)=10.82026.03 | 62.3 | 40 | 51 | 44.5 | 33.8 | 39.2 | 37.3 | 40.3 | 38.8 | 40.6 | 42.7 | |
| Gemini Pro 2.5Projection format=CMP, Inf. Time (seconds)=18.32026.03 | 61.5 | 44.8 | 53 | 52.2 | 46.7 | 49.5 | 46.8 | 36.9 | 41.9 | 37.2 | 46.5 | |
| ZoomEyeProjection format=ERP, Inf. Time (seconds)=19.52026.03 | 61.5 | 26.6 | 43.9 | 44.2 | 31.9 | 38.1 | 25.3 | 31.3 | 28.3 | 28.5 | 35.5 | |
| Free360Projection format=Hybrid, Inf. Time (seconds)=22.52026.03 | 60.7 | 30 | 45.2 | 54.7 | 43.9 | 49.3 | 33.6 | 53.3 | 43.6 | 41.4 | 45.3 | |
| Qwen2.5-VL-7BProjection format=ERP, Inf. Time (seconds)=2.12026.03 | 59.5 | 27.9 | 43.6 | 42.2 | 36 | 39.1 | 30.7 | 31.2 | 31 | 33.7 | 37.3 | |
| ZoomEyeProjection format=CMP, Inf. Time (seconds)=23.92026.03 | 59.2 | 22.8 | 40.8 | 51.2 | 30.3 | 40.7 | 27.3 | 29.1 | 28.2 | 28 | 35.3 | |
| Omni-CoTProjection format=Hybrid, Inf. Time (seconds)=16.62026.03 | 58.9 | 24.3 | 41.4 | 49.5 | 40.3 | 44.9 | 28.3 | 41 | 34.7 | 35 | 39.5 | |
| Qwen2.5-VL-7BProjection format=CMP, Inf. Time (seconds)=2.22026.03 | 58.1 | 27 | 42.4 | 50.6 | 42 | 46.3 | 28.4 | 30.4 | 29.4 | 30.6 | 38.1 | |
| Gemini Flash 2.5Projection format=CMP, Inf. Time (seconds)=12.52026.03 | 57.5 | 41.9 | 49.6 | 56.5 | 46 | 51.3 | 37.1 | 25.8 | 31.5 | 34.7 | 42.6 | |
| DeepEyesProjection format=ERP, Inf. Time (seconds)=78.42026.03 | 53.6 | 25.2 | 39.3 | 33.3 | 32 | 32.7 | 28.2 | 26.9 | 27.5 | 29.1 | 32.6 | |
| DeepEyesProjection format=CMP, Inf. Time (seconds)=66.92026.03 | 52.6 | 25.6 | 39 | 43.5 | 42.5 | 43 | 31.4 | 26.3 | 28.9 | 27.3 | 35.5 | |
| GPT-4oProjection format=ERP, Inf. Time (seconds)=1.52026.03 | 48.6 | 32 | 40.3 | 54.2 | 38.8 | 46.5 | 37.7 | 35.7 | 36.7 | 38.5 | 40.7 | |
| GPT-4oProjection format=CMP, Inf. Time (seconds)=1.62026.03 | 45.5 | 30.8 | 38.1 | 63 | 44 | 53.5 | 37.4 | 34 | 35.7 | 35.3 | 41.3 | |
| DeepSeek-VL2-SmallProjection format=ERP, Inf. Time (seconds)=0.92026.03 | 41.8 | 22.7 | 32.2 | 42.4 | 39.3 | 40.9 | 27.2 | 32.6 | 29.9 | 32.4 | 33.9 | |
| DeepSeek-VL2-SmallProjection format=CMP, Inf. Time (seconds)=0.92026.03 | 40.3 | 19.2 | 29.7 | 55.4 | 43.1 | 49.2 | 28.2 | 29.7 | 28.9 | 31.1 | 35.1 | |
| DC2Projection format=ERP, Inf. Time (seconds)=617.92026.03 | 40 | 16 | 27.9 | 44 | 36.4 | 40.2 | 22 | 25.1 | 23.6 | 21.5 | 29.1 | |
| DC2Projection format=CMP, Inf. Time (seconds)=761.62026.03 | 39.7 | 11 | 25.2 | 49.1 | 34.9 | 42 | 20.2 | 23.1 | 21.6 | 20.2 | 28.1 | |
| LLaVA-CoTProjection format=ERP, Inf. Time (seconds)=2.72026.03 | 34.9 | 16.7 | 25.7 | 41.5 | 33.3 | 37.4 | 27.2 | 30 | 28.6 | 27.5 | 30.1 | |
| SEALProjection format=CMP, Inf. Time (seconds)=14.62026.03 | 33.6 | 21.4 | 27.5 | 32.1 | 30.4 | 31.3 | 23.6 | 18.7 | 21.1 | 20.7 | 25.7 | |
| LLaVA-CoTProjection format=CMP, Inf. Time (seconds)=3.02026.03 | 33.3 | 16.9 | 25 | 55.6 | 37.3 | 46.5 | 24.8 | 30.9 | 27.8 | 27.6 | 32.2 | |
| LLaVA-v1.5-7BProjection format=CMP, Inf. Time (seconds)=0.52026.03 | 32.5 | 9.3 | 20.8 | 33.4 | 39.6 | 36.5 | 23.5 | 23.9 | 23.7 | 26.3 | 26.8 | |
| SEALProjection format=ERP, Inf. Time (seconds)=14.62026.03 | 31.3 | 24.6 | 27.9 | 30.4 | 27.8 | 29.1 | 23.6 | 20.4 | 22 | 18.9 | 25.2 | |
| LLaVA-v1.5-7BProjection format=ERP, Inf. Time (seconds)=0.42026.03 | 30.7 | 8.4 | 19.4 | 29.3 | 35.9 | 32.6 | 22.4 | 26.7 | 24.5 | 26.6 | 25.6 | |
| VisCoT-7B-336Projection format=CMP, Inf. Time (seconds)=0.82026.03 | 28.3 | 23.7 | 26 | 34.8 | 36.4 | 35.6 | 24.6 | 24.4 | 24.5 | 24.2 | 28 | |
| VisCoT-7B-336Projection format=ERP, Inf. Time (seconds)=1.12026.03 | 28.1 | 23.3 | 25.7 | 30.6 | 35.9 | 33.3 | 22.3 | 26.7 | 24.4 | 23.8 | 27.2 | |
| Random guessProjection format=—2026.03 | 25 | 25 | 25 | 25 | 25 | 25 | 25 | 25 | 25 | 25 | 25 |