Trajectory-based Video Question Answering on STEMO-Bench
77.1Accuracy (Target)STEMO-Track (Gemini-3-Flash)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| STEMO-Track (Gemini-3-Flash)Backbone=Gemini-3-Flash2026.05 | 77.1 | 76.3 | 81.3 | |
| STEMO-Track (Qwen3)Backbone=Qwen32026.05 | 74 | 72.6 | 78.1 | |
| Qwen3.5-27BType=Open-weight end-to-end MLLM2026.05 | 68.4 | 66.7 | 73.3 | |
| Qwen3-VL-32B ThinkType=Open-weight end-to-end MLLM2026.05 | 65 | 70.6 | 78.1 | |
| Gemini-3.1-ProType=Proprietary end-to-end MLLM2026.05 | 61.4 | 64.9 | 69.2 | |
| Qwen3-VL-8B ThinkType=Open-weight end-to-end MLLM2026.05 | 61.3 | 70.1 | 77.4 | |
| Qwen3-VL-32B InstructType=Open-weight end-to-end MLLM2026.05 | 59.8 | 70 | 78.1 | |
| Gemma-4-27BType=Proprietary end-to-end MLLM2026.05 | 55.5 | 60 | 72.7 | |
| Cosmos-Reason2-8BType=Open-weight end-to-end MLLM2026.05 | 54.7 | 74.7 | 83.7 | |
| SeViLAType=Structured pipeline2026.05 | 51.4 | 59.6 | 56.8 | |
| TraveLERType=Structured pipeline2026.05 | 50.6 | 41.8 | 49.3 | |
| Qwen3.5-9BType=Open-weight end-to-end MLLM2026.05 | 50.5 | 49 | 57.9 | |
| VideoMind-7BType=Structured pipeline2026.05 | 50.3 | 37.9 | 43.2 | |
| InternVL3-78BType=Open-weight end-to-end MLLM2026.05 | 50 | 66.1 | 71 | |
| Gemini-3-FlashType=Proprietary end-to-end MLLM2026.05 | 48.7 | 61.7 | 69.9 | |
| Claude-4.6-SonnetType=Proprietary end-to-end MLLM2026.05 | 48.3 | 62.9 | 73.2 | |
| VideoRFT-7BType=Open-weight end-to-end MLLM2026.05 | 38.6 | 55.1 | 50.8 | |
| Qwen3-VL-8B InstructType=Open-weight end-to-end MLLM2026.05 | 34.3 | 59.8 | 77.4 | |
| GPT-5Type=Proprietary end-to-end MLLM2026.05 | 32.5 | 43.2 | 70.5 | |
| AKSType=Budget-aware pipeline, Backbone=Qwen3-VL-8B2026.05 | 27.6 | 53.9 | 75.6 | |
| CLIP-RetrievalType=Budget-aware pipeline, Backbone=Qwen3-VL-8B2026.05 | 26.2 | 52.9 | 72.8 | |
| Video-R1-7BType=Open-weight end-to-end MLLM2026.05 | 24.8 | 19.6 | 21.9 | |
| EFSType=Budget-aware pipeline, Backbone=Qwen3-VL-8B2026.05 | 24.5 | 50.2 | 73.1 |