Streaming Video Understanding on OVO-Bench 1.0 (test)
89.9OCRAURA
Evaluation results
| Method | Links | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| AURAModel Category=Open-Source2026.04 | 89.9 | 79.8 | 80.2 | 70.2 | 77.2 | 81.5 | 79.8 | 54.9 | 67.6 | 58.6 | 60.4 | 30.4 | 75.8 | 61.3 | 55.8 | 65.3 | |
| Gemini-1.5-ProModel Category=Proprietary2026.04 | 85.9 | 67 | 79.3 | 58.4 | 63.4 | 62 | 69.3 | 58.6 | 76.4 | 52.6 | 62.5 | 35.5 | 74.2 | 61.7 | 57.2 | 63 | |
| Streamo-7BModel Category=Open-Source2026.04 | 77.2 | 66.1 | 76.7 | 45.5 | 66.3 | 72.8 | 67.4 | 55.6 | 58.1 | 33.9 | 49.2 | 30.8 | 57.6 | 82.5 | 57 | 57.9 | |
| ViSpeakModel Category=Open-Source2026.04 | 75.2 | 58.7 | 71.6 | 51.1 | 74.3 | 66.9 | 66.3 | 59.9 | 48.7 | 64 | 57.5 | 33.8 | 68.5 | 60.4 | 54.3 | 61.1 | |
| MiniCPM-o-4.5Model Category=Open-Source2026.04 | 73.2 | 65.1 | 75 | 59 | 69.3 | 62.5 | 67.3 | 55.2 | 66.9 | 45.7 | 55.9 | 43 | 66 | 58.8 | 55.9 | 59.7 | |
| StreamAgentModel Category=Open-Source2026.04 | 71.2 | 53.2 | 63.6 | 53.9 | 67.3 | 58.7 | 61.3 | 54.8 | 58.1 | 25.8 | 41.7 | 35.9 | 48.4 | 52 | 45.4 | 49.4 | |
| GPT-4oModel Category=Proprietary2026.04 | 69.8 | 64.2 | 71.6 | 51.1 | 70.3 | 59.8 | 64.5 | 57.9 | 75.7 | 48.7 | 60.8 | 73.2 | 27.6 | 59.4 | 53.4 | 59.5 | |
| Qwen3-VL-8B-Inst.Model Category=Open-Source2026.04 | 54.5 | 58.7 | 68.5 | 70.3 | 68.1 | 49.5 | 61.6 | 58.8 | 41.1 | 23.1 | 41 | 21.2 | 51.3 | 40.4 | 37.7 | 46.8 | |
| Streamo-4B (Qwen3-VL)# Frames=1fps, Backbone=Qwen3-VL-4B2025.12 | 0.8255 | 0.6972 | 0.7414 | 0.5225 | 0.7327 | 0.8152 | 0.7224 | 0.5819 | 0.527 | 0.172 | 0.427 | 0.3138 | 0.539 | 0.8417 | 0.5648 | 0.551 | |
| Streamo-7B (Qwem2.5-VL)# Frames=1fps, Backbone=Qwen2.5-VL-7B2025.12 | 0.7919 | 0.578 | 0.75 | 0.4944 | 0.6436 | 0.7011 | 0.6598 | 0.5455 | 0.5203 | 0.3172 | 0.461 | 0.2996 | 0.5103 | 0.8333 | 0.5477 | 0.5561 | |
| Streamo-3B (Qwem2.5-VL)# Frames=1fps, Backbone=Qwen2.5-VL-3B2025.12 | 0.7852 | 0.5229 | 0.6724 | 0.4438 | 0.5545 | 0.712 | 0.6151 | 0.5118 | 0.5743 | 0.1667 | 0.4176 | 0.2794 | 0.5072 | 0.825 | 0.5372 | 0.5233 | |
| Streamo-2B (InternVL3)# Frames=1fps, Backbone=InternVL3-2B2025.12 | 0.7718 | 0.5596 | 0.6207 | 0.4101 | 0.604 | 0.7011 | 0.6112 | 0.4882 | 0.473 | 0.1344 | 0.3652 | 0.2923 | 0.4738 | 0.8042 | 0.5234 | 0.4999 | |
| LLaVA-Video-7B# Frames=642025.12 | 0.6913 | 0.5872 | 0.6883 | 0.4944 | 0.7426 | 0.5978 | 0.6352 | 0.5623 | 0.5743 | 0.0753 | 0.404 | 0.341 | 0.6995 | 0.6042 | 0.5482 | 0.5291 | |
| InternVL-V2-8B# Frames=642025.12 | 0.6711 | 0.6055 | 0.6379 | 0.4607 | 0.6832 | 0.5652 | 0.6039 | 0.4815 | 0.5743 | 0.2473 | 0.4344 | 0.265 | 0.5914 | 0.5414 | 0.466 | 0.5015 | |
| LLaVA-OneVision-7B# Frames=642025.12 | 0.6644 | 0.578 | 0.7328 | 0.5337 | 0.7129 | 0.6196 | 0.6402 | 0.5421 | 0.5541 | 0.2151 | 0.4371 | 0.2564 | 0.6709 | 0.5875 | 0.505 | 0.5274 | |
| Qwen2-VL-72B# Frames=642025.12 | 0.6577 | 0.6055 | 0.6983 | 0.5169 | 0.6931 | 0.5435 | 0.6192 | 0.5253 | 0.6081 | 0.5753 | 0.5695 | 0.3883 | 0.6407 | 0.45 | 0.493 | 0.5627 | |
| Qwen2-VL-7B# Frames=642025.12 | 0.604 | 0.5046 | 0.5603 | 0.4719 | 0.6634 | 0.5543 | 0.5598 | 0.4781 | 0.3548 | 0.5608 | 0.4646 | 0.3166 | 0.6582 | 0.4875 | 0.4874 | 0.5039 | |
| Dispider-7B# Frames=1fps2025.12 | 0.5772 | 0.4954 | 0.6207 | 0.4494 | 0.6139 | 0.5163 | 0.5455 | 0.4848 | 0.5541 | 0.043 | 0.3606 | 0.1805 | 0.3736 | 0.4875 | 0.3472 | 0.4178 | |
| LongVU-7B# Frames=1fps2025.12 | 0.5369 | 0.5321 | 0.6293 | 0.4775 | 0.6832 | 0.5978 | 0.5761 | 0.4074 | 0.5946 | 0.0484 | 0.3501 | 0.1218 | 0.6948 | 0.6083 | 0.475 | 0.4671 | |
| Flash-VStream-7B# Frames=1fps2025.12 | 0.2416 | 0.2936 | 0.2845 | 0.3371 | 0.2574 | 0.288 | 0.2837 | 0.3906 | 0.3716 | 0.0591 | 0.2738 | 0.0802 | 0.6725 | 0.6 | 0.4509 | 0.3361 | |
| VideoLLM-online-8B# Frames=2fps2025.12 | 0.0805 | 0.2385 | 0.1207 | 0.1404 | 0.4554 | 0.212 | 0.2079 | 0.2222 | 0.188 | 0.1218 | 0.1773 | — | — | — | — | — |