Long Video Question Answering on MovieChat-1K Global Mode (test)
87.5AccuracyHierarQ
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| HierarQLLM finetuning=true2025.03 | 87.5 | 4.7 | |
| HierarQLLM finetuning=false2025.03 | 86.9 | 4.7 | |
| FVS+S32025.03 | 84 | 4.6 | |
| ReWindNum Frames=548*, Num Tokens=1184*, Evaluation Protocol=Zero-shot GPT-3.5, Frame Rate=1fps2024.11 | 80.6 | 4.46 | |
| MovieChat+Text Decoder=LLM based, # Frames=20482024.04 | 71.2 | 3.51 | |
| MovieChatNum Frames=2048, Num Tokens=8192, Evaluation Protocol=Zero-shot GPT-3.52024.11 | 67.8 | 3.81 | |
| MovieChat# Frames=20482023.07 | 62.3 | 3.23 | |
| MovieChatText Decoder=LLM based, # Frames=20482024.04 | 62.3 | 3.23 | |
| MovieChat2025.03 | 62.3 | 3.2 | |
| MA-LMMreproduced results=true2025.03 | 61.4 | 3.2 | |
| Video ChatNum Frames=32, Num Tokens=3072, Evaluation Protocol=Zero-shot GPT-3.52024.11 | 61 | 3.34 | |
| Video Chat# Frames=322023.07 | 57.8 | 3 | |
| Video ChatText Decoder=LLM based, # Frames=322024.04 | 57.8 | 3 | |
| Video Chat2025.03 | 57.8 | 3 | |
| Video LLaMA# Frames=322023.07 | 51.7 | 2.67 | |
| Video LLAMAText Decoder=LLM based, # Frames=322024.04 | 51.7 | 2.67 | |
| Video LLaMA2025.03 | 51.7 | 2.7 | |
| Video LLAMANum Frames=32, Num Tokens=32, Evaluation Protocol=Zero-shot GPT-3.52024.11 | 51.4 | 3.1 | |
| Video-ChatGPT# Frames=1002023.07 | 47.6 | 2.55 | |
| Video-ChatGPTText Decoder=LLM based, # Frames=1002024.04 | 47.6 | 2.55 | |
| Video-ChatGPT2025.03 | 47.6 | 2.6 | |
| Video-ChatGPTNum Frames=100, Num Tokens=356, Evaluation Protocol=Zero-shot GPT-3.52024.11 | 44.2 | 2.71 | |
| mPLUG-2Text Decoder=non-LLM based, # Frames=82024.04 | 31.7 | 2.13 | |
| GITText Decoder=non-LLM based, # Frames=62024.04 | 28.8 | 1.83 |