List your company
ResearchFormerly WizWand

Long Video Question Answering on MovieChat-1K Global Mode (test)

87.5AccuracyHierarQ
26.45242.30158.1573.999Jul 31, 2023Nov 6, 2023Feb 12, 2024May 20, 2024Aug 26, 2024Dec 2, 2024Mar 11, 2025
Updated 8mo ago

Evaluation results

MethodLinks
HierarQLLM finetuning=true2025.0387.54.7
HierarQLLM finetuning=false2025.0386.94.7
FVS+S32025.03844.6
ReWindNum Frames=548*, Num Tokens=1184*, Evaluation Protocol=Zero-shot GPT-3.5, Frame Rate=1fps2024.1180.64.46
MovieChat+Text Decoder=LLM based, # Frames=20482024.0471.23.51
MovieChatNum Frames=2048, Num Tokens=8192, Evaluation Protocol=Zero-shot GPT-3.52024.1167.83.81
MovieChat# Frames=20482023.0762.33.23
MovieChatText Decoder=LLM based, # Frames=20482024.0462.33.23
MovieChat2025.0362.33.2
MA-LMMreproduced results=true2025.0361.43.2
Video ChatNum Frames=32, Num Tokens=3072, Evaluation Protocol=Zero-shot GPT-3.52024.11613.34
Video Chat# Frames=322023.0757.83
Video ChatText Decoder=LLM based, # Frames=322024.0457.83
Video Chat2025.0357.83
Video LLaMA# Frames=322023.0751.72.67
Video LLAMAText Decoder=LLM based, # Frames=322024.0451.72.67
Video LLaMA2025.0351.72.7
Video LLAMANum Frames=32, Num Tokens=32, Evaluation Protocol=Zero-shot GPT-3.52024.1151.43.1
Video-ChatGPT# Frames=1002023.0747.62.55
Video-ChatGPTText Decoder=LLM based, # Frames=1002024.0447.62.55
Video-ChatGPT2025.0347.62.6
Video-ChatGPTNum Frames=100, Num Tokens=356, Evaluation Protocol=Zero-shot GPT-3.52024.1144.22.71
mPLUG-2Text Decoder=non-LLM based, # Frames=82024.0431.72.13
GITText Decoder=non-LLM based, # Frames=62024.0428.81.83