Online Video Understanding on OVBench 1.0 (test)
71.4AAGemini-1.5-Flash
Evaluation Results
| Method | Links | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Gemini-1.5-FlashEvaluation Protocol=sliding window, sliding window size=32s, fps=22024.12 | 71.4 | 53.6 | 21.9 | 56.5 | 60.8 | — | 36.7 | 47.9 | 62.5 | 32.3 | 87 | 50 | 83.3 | 22.3 | 46.9 | 50.7 | |
| LLaVA-OnevisionSize=7B, Evaluation Protocol=sliding window, sliding window size=32s, fps=22024.12 | 68 | 62.7 | 35.9 | 58.4 | 50.3 | — | 29.4 | 60.7 | 58 | 43.1 | 86.5 | 49.7 | 70.7 | 28.1 | 30.2 | 49.5 | |
| LongVASize=7B, Evaluation Protocol=sliding window, sliding window size=32s, fps=22024.12 | 64.1 | 56.5 | 29.5 | 54.9 | 51.9 | — | 35.3 | 55.6 | 57.7 | 31.6 | 67.4 | 44.7 | 80 | 26.7 | 4 | 43.6 | |
| VideoChat-OnlineSize=4B, Evaluation Protocol=Streaming video input, fps=22024.12 | 64.1 | 59.7 | 16.6 | 63.1 | 58.3 | — | 42.2 | 54.4 | 70.6 | 54.1 | 88.7 | 48.5 | 73 | 25.9 | 71.7 | 54.9 | |
| Qwen2-VLSize=7B, Evaluation Protocol=sliding window, sliding window size=32s, fps=22024.12 | 60.3 | 66.1 | 22.1 | 54.9 | 51.5 | — | 37.8 | 64.4 | 69.3 | 35.3 | 97 | 49.4 | 65.1 | 30.8 | 11.7 | 49.7 | |
| InternVL2Size=4B, Evaluation Protocol=sliding window, sliding window size=32s, fps=22024.12 | 57.7 | 57 | 14.4 | 59.2 | 49.4 | — | 30.3 | 61.8 | 46.3 | 30.9 | 83 | 32.3 | 70.7 | 29.4 | 3.4 | 44.1 | |
| VideoChat-OnlineSize=4B, Evaluation Protocol=sliding window, sliding window size=32s, fps=22024.12 | 56.4 | 63 | 15.6 | 57.1 | 57.9 | — | 39.1 | 54.2 | 73.9 | 41.3 | 92.2 | 53.1 | 69.8 | 27.3 | 69.9 | 53.9 | |
| InternVL2Size=7B, Evaluation Protocol=sliding window, sliding window size=32s, fps=22024.12 | 52.6 | 60.2 | 27.6 | 57.5 | 52 | — | 38.8 | 67.1 | 58.3 | 38.1 | 87.4 | 37 | 75.4 | 31.4 | 5.9 | 48.7 | |
| LLaMA-VIDSize=7B, Evaluation Protocol=sliding window, sliding window size=32s, fps=22024.12 | 43.6 | 50.9 | 19.6 | 64 | 47.5 | — | 29.4 | 48.9 | 51.2 | 31.9 | 75.7 | 24.8 | 59.1 | 26 | 40 | 41.9 | |
| VTimeLLMSize=7B, Evaluation Protocol=sliding window, sliding window size=32s, fps=22024.12 | 37.2 | 23.4 | 15 | 64.8 | 43.8 | — | 25.9 | 38.8 | 32.5 | 25.9 | 40.9 | 6.8 | 48.4 | 43.5 | 8.6 | 33.1 | |
| MiniCPM-V2.6Size=7B, Evaluation Protocol=sliding window, sliding window size=32s, fps=22024.12 | 33.3 | 35.9 | 15 | 59.2 | 50.8 | — | 25 | 37.4 | 41.7 | 26.6 | 98.3 | 36.3 | 66.1 | 26.4 | 6.2 | 39.1 | |
| Flash-VstreamSize=7B, Evaluation Protocol=Streaming video input, fps=22024.12 | 26.9 | 37.6 | 23.9 | 60.1 | 41.9 | — | 23.4 | 35.3 | 26.1 | 24.7 | 27 | 21.4 | 29.8 | 25.6 | 26.8 | 31.2 | |
| MovieChatSize=7B, Evaluation Protocol=Streaming video input, fps=22024.12 | 23.1 | 27.5 | 23.6 | 58.4 | 43.9 | — | 25.6 | 31.1 | 23.9 | 26.9 | 24.4 | 28.9 | 29.3 | 25.5 | 21.9 | 30.9 | |
| LITASize=7B, Evaluation Protocol=sliding window, sliding window size=32s, fps=22024.12 | 19.2 | 24.5 | 19.9 | 40.8 | 48.9 | — | 3.1 | 27.3 | 6.4 | 6.9 | 35.2 | 23.9 | 27.4 | 0.5 | 3.4 | 20.4 | |
| TimeChatSize=7B, Evaluation Protocol=sliding window, sliding window size=32s, fps=22024.12 | 7.7 | 15.3 | 18.7 | 20.6 | 15.7 | — | 9.1 | 14.7 | 9.8 | 7.5 | 13.9 | 10.3 | 9.3 | 10.1 | 10.8 | 12.8 | |
| VideoLLM-OnlineSize=7B, Evaluation Protocol=Streaming video input, fps=22024.12 | 0 | 1.8 | 20.9 | 5.2 | 5.9 | — | 0 | 2.3 | 26.7 | 0.6 | 0.9 | 19.9 | 0.9 | 1.7 | 8.3 | 9.6 |