List your company

Spatio-temporal reasoning on VSTemporalI-Bench

36.1Object-Object Relative Position ErrorChance Level (Random)
33.64450.22266.883.378May 26, 2025

Evaluation results

MethodLinks
Chance Level (Random)Rank=-2025.0536.1———36.150
VILA-1.5-8BRank=8, Model Type=Open-sourced VLMs2025.0550.437.330.127.342.236.7
Chance Level (Frequency)Rank=-2025.0552.227.45.46.240.732.4
LongVILA-8BRank=11, Model Type=Open-sourced VLMs2025.0552.330.52011.635.433.4
LongVA-7BRank=10, Model Type=Open-sourced VLMs2025.0557.932.313.55.143.741.2
GPT-4oRank=2, Model Type=Proprietary Models (API)2025.0558.138.229.523.437.342.5
LLaVA-OneVision-7BRank=4, Model Type=Open-sourced VLMs2025.0562.141.729.919.347.549.8
LLaVA-NeXT-Video-7BRank=5, Model Type=Open-sourced VLMs2025.0564.74028.21.849.855.6
VILA-1.5-40BRank=6, Model Type=Open-sourced VLMs2025.0565.438.228.215.728.853
InternVL2-8BRank=3, Model Type=Open-sourced VLMs2025.056843.532.913.54855
LLaVA-NeXT-Video-72BRank=2, Model Type=Open-sourced VLMs2025.0578.34432.310.548.150.9
Gemini-2.5 ProRank=1, Model Type=Proprietary Models (API)2025.058442.429.283060.7
VLM-3R (7B)Rank=1, Model Type=Open-sourced VLMs2025.0586.558.839.439.660.668.6
†Human LevelRank=-2025.0597.57751.446.895.194.3