Video Question Answering on MovieChat-1k Breakpoint
74.72AccuracyVideo-QTR
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Video-QTRFrames=202.39*2025.12 | 74.72 | — | |
| HERMESEvaluation Protocol=Fully Supervised2024.08 | 65.8 | 3.65 | |
| HERMESEvaluation Protocol=Zero-shot2024.08 | 57.3 | 3.29 | |
| qwen2.5-vl-maxFrames=5122025.12 | 56.59 | — | |
| DrVideoArchitecture=Based on Open-source Captioners and Proprietary LLMs2024.06 | 56.4 | 2.75 | |
| gemini2.5-proFrames=10242025.12 | 54.76 | — | |
| MovieChat+Architecture=Open-Source MLLM2024.06 | 49.6 | 2.62 | |
| MovieChat+Frames=20482025.12 | 49.6 | — | |
| MovieChatArchitecture=Open-Source MLLM2024.06 | 48.3 | 2.46 | |
| MovieChatEvaluation Protocol=Zero-shot2024.08 | 48.1 | 2.46 | |
| Video-ChatGPTFrames=1002025.12 | 48.1 | — | |
| Video-ChatGPTArchitecture=Open-Source MLLM2024.06 | 48 | 2.43 | |
| Video-ChatGPTEvaluation Protocol=Zero-shot2024.08 | 47.8 | 2.43 | |
| VideoChatEvaluation Protocol=Zero-shot2024.08 | 46.3 | 2.32 | |
| VideoChatFrames=642025.12 | 46.11 | — | |
| Video ChatArchitecture=Open-Source MLLM2024.06 | 46.1 | 2.32 | |
| Video-LLaMAEvaluation Protocol=Zero-shot2024.08 | 45.8 | 2.11 | |
| Video LLaMAArchitecture=Open-Source MLLM2024.06 | 39.1 | 2.11 | |
| VideoLLaMAFrames=322025.12 | 39.1 | — | |
| VideoAgentArchitecture=Based on Open-source Captioners and Proprietary LLMs, re-implemented=true2024.06 | 31.6 | 2.05 | |
| mPLUG-2Frames=82025.12 | 30.81 | — | |
| LLoViArchitecture=Based on Open-source Captioners and Proprietary LLMs2024.06 | 17.8 | 1.03 | |
| MM-VIDArchitecture=Based on Open-source Captioners and Proprietary LLMs2024.06 | 10.4 | 0.56 |