Long Video Question Answering on MovieChat-1K Breakpoint Mode (test)
76.4AccuracyHierarQ
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| HierarQLLM finetuning=true2025.03 | 76.4 | 4.2 | |
| HierarQLLM finetuning=false2025.03 | 74.2 | 4.1 | |
| FVS+S32025.03 | 73.5 | 4 | |
| ReWindNum Frames=548*, Num Tokens=1184*, Evaluation Protocol=Zero-shot GPT-3.5, Frame Rate=1fps2024.11 | 57.2 | 3.4 | |
| MovieChatNum Frames=2048, Num Tokens=8192, Evaluation Protocol=Zero-shot GPT-3.52024.11 | 50.4 | 2.96 | |
| MA-LMMreproduced results=true2025.03 | 50.4 | 2.7 | |
| Video-ChatGPTNum Frames=100, Num Tokens=356, Evaluation Protocol=Zero-shot GPT-3.52024.11 | 49.8 | 2.71 | |
| MovieChat+Text Decoder=LLM based, # Frames=20482024.04 | 49.6 | 2.62 | |
| MovieChat# Frames=20482023.07 | 48.3 | 2.57 | |
| MovieChatText Decoder=LLM based, # Frames=20482024.04 | 48.3 | 2.57 | |
| Video ChatNum Frames=32, Num Tokens=3072, Evaluation Protocol=Zero-shot GPT-3.52024.11 | 48.3 | 2.43 | |
| MovieChat2025.03 | 48.3 | 2.6 | |
| Video-ChatGPT# Frames=1002023.07 | 48 | 2.45 | |
| Video-ChatGPTText Decoder=LLM based, # Frames=1002024.04 | 48 | 2.45 | |
| Video-ChatGPT2025.03 | 48 | 2.5 | |
| Video Chat# Frames=322023.07 | 46.1 | 2.29 | |
| Video ChatText Decoder=LLM based, # Frames=322024.04 | 46.1 | 2.29 | |
| Video Chat2025.03 | 46.1 | 2.3 | |
| Video LLaMA# Frames=322023.07 | 39.1 | 2.04 | |
| Video LLAMAText Decoder=LLM based, # Frames=322024.04 | 39.1 | 2.04 | |
| Video LLaMA2025.03 | 39.1 | 2 | |
| Video LLAMANum Frames=32, Num Tokens=32, Evaluation Protocol=Zero-shot GPT-3.52024.11 | 38.2 | 2.31 | |
| mPLUG-2Text Decoder=non-LLM based, # Frames=82024.04 | 30.8 | 1.83 | |
| GITText Decoder=non-LLM based, # Frames=62024.04 | 29.2 | 1.98 |