List your company
ResearchFormerly WizWand

Long Video Question Answering on MovieChat-1K Breakpoint Mode (test)

76.4AccuracyHierarQ
27.31240.05652.865.544Jul 31, 2023Nov 6, 2023Feb 12, 2024May 20, 2024Aug 26, 2024Dec 2, 2024Mar 11, 2025
Updated 8mo ago

Evaluation results

MethodLinks
HierarQLLM finetuning=true2025.0376.44.2
HierarQLLM finetuning=false2025.0374.24.1
FVS+S32025.0373.54
ReWindNum Frames=548*, Num Tokens=1184*, Evaluation Protocol=Zero-shot GPT-3.5, Frame Rate=1fps2024.1157.23.4
MovieChatNum Frames=2048, Num Tokens=8192, Evaluation Protocol=Zero-shot GPT-3.52024.1150.42.96
MA-LMMreproduced results=true2025.0350.42.7
Video-ChatGPTNum Frames=100, Num Tokens=356, Evaluation Protocol=Zero-shot GPT-3.52024.1149.82.71
MovieChat+Text Decoder=LLM based, # Frames=20482024.0449.62.62
MovieChat# Frames=20482023.0748.32.57
MovieChatText Decoder=LLM based, # Frames=20482024.0448.32.57
Video ChatNum Frames=32, Num Tokens=3072, Evaluation Protocol=Zero-shot GPT-3.52024.1148.32.43
MovieChat2025.0348.32.6
Video-ChatGPT# Frames=1002023.07482.45
Video-ChatGPTText Decoder=LLM based, # Frames=1002024.04482.45
Video-ChatGPT2025.03482.5
Video Chat# Frames=322023.0746.12.29
Video ChatText Decoder=LLM based, # Frames=322024.0446.12.29
Video Chat2025.0346.12.3
Video LLaMA# Frames=322023.0739.12.04
Video LLAMAText Decoder=LLM based, # Frames=322024.0439.12.04
Video LLaMA2025.0339.12
Video LLAMANum Frames=32, Num Tokens=32, Evaluation Protocol=Zero-shot GPT-3.52024.1138.22.31
mPLUG-2Text Decoder=non-LLM based, # Frames=82024.0430.81.83
GITText Decoder=non-LLM based, # Frames=62024.0429.21.98