Open-ended Video Question Answering on MSVD-QA
79.9AccuracyPLLaVA
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| PLLaVALLM Size=34B, Vision Encoder=CLIP-L2024.07 | 79.9 | 4.2 | |
| SF-LLaVA-34BLLM Size=34B, Vision Encoder=CLIP-L2024.07 | 79.9 | 4.1 | |
| PLLAVALLM Size=34B, Vision Encoder=CLIP-L, Training-free=false2024.11 | 79.9 | 4.2 | |
| SF-LLAVALLM Size=34B, Vision Encoder=CLIP-L, Training-free=true2024.11 | 79.9 | 4.1 | |
| IG-VLM (LLaVA-v1.6)LLM Size=34B, Vision Encoder=CLIP-L2024.07 | 79.6 | 4.1 | |
| IG-VLMLLM Size=34B, Vision Encoder=CLIP-L, Training-free=true2024.11 | 79.6 | 4.1 | |
| TS-LLAVALLM Size=34B, Vision Encoder=CLIP-L, Training-free=true2024.11 | 79.4 | 4.1 | |
| SF-LLaVALLM Size=7B, Vision Encoder=CLIP-L, Training-free=true2024.11 | 79.1 | 4.1 | |
| TS-LLaVALLM Size=7B, Vision Encoder=CLIP-L, Training-free=true2024.11 | 79 | 4.1 | |
| IG-VLMLLM Size=7B, Vision Encoder=CLIP-L, Training-free=true2024.11 | 78.8 | 4.1 | |
| PLLAVALLM Size=7B, Vision Encoder=CLIP-L, Training-free=false2024.11 | 76.6 | 4.1 | |
| P-LLaVAParameters=7B2025.01 | 76.6 | 4.1 | |
| MovieChat+LLM Size=7B, Vision Encoder=CLIP-G, Training-free=false2024.11 | 76.5 | 3.9 | |
| Grounded-VideoLLM2025.01 | 76.3 | 4.1 | |
| DeepStack-LLLM Size=7B, Vision Encoder=CLIP-L, Training-free=true2024.11 | 76 | 4 | |
| LLaVA-ST2025.01 | 75.9 | 4.1 | |
| Elysium2025.01 | 75.8 | 3.7 | |
| MovieChatLLM Size=7B, Vision Encoder=CLIP-G, Training-free=false2024.11 | 75.2 | 3.8 | |
| MovieChat2025.01 | 75.2 | 3.8 | |
| ST-LLM2025.01 | 74.6 | 3.9 | |
| FreeVALLM Size=7B, Vision Encoder=CLIP-L, Training-free=true2024.11 | 73.8 | 4.1 | |
| VideoGPT+LLM Size=3.8B, Vision Encoder=CLIP-L, Training-free=false2024.11 | 72.4 | 3.9 | |
| Baichuan-omni (7B)# Frames=1 fps (max 48)2024.10 | 72.2 | 4 | |
| Video-LLaMA2LLM Size=7B, Vision Encoder=CLIP-L, Training-free=false2024.11 | 70.9 | 3.8 | |
| VideoLLaMA 2 (7B)# Frames=162024.10 | 70.9 | 3.8 | |
| Video-LLaVALLM Size=7B, Vision Encoder=ViT-L, Training-free=false2024.11 | 70.7 | 3.9 | |
| Video-LLaVA (7B)# Frames=82024.10 | 70.7 | 3.9 | |
| Video-LLaMA2LLM Size=46.7B, Vision Encoder=CLIP-L2024.07 | 70.5 | 3.8 | |
| Video-LLaMA2LLM Size=46.7B, Vision Encoder=CLIP-L, Training-free=false2024.11 | 70.5 | 3.8 | |
| VideoChat2LLM Size=7B, Vision Encoder=UMT-L, Training-free=false2024.11 | 70 | 3.9 | |
| VideoChat2 (7B)# Frames=162024.10 | 70 | 3.9 | |
| LongVLM2025.01 | 70 | 3.8 | |
| VideoChat22025.01 | 70 | 3.9 | |
| LLaMA-VIDLLM Size=13B, Vision Encoder=CLIP-G, Training-free=false2024.11 | 69.7 | 3.7 | |
| Momentor2025.01 | 68.9 | 3.6 | |
| GroundingGPT2025.01 | 67.8 | 3.7 | |
| LLaVA-NeXT-Video (7B)# Frames=322024.10 | 67.4 | 3.4 | |
| Vista-LLaMALLM Size=7B, Vision Encoder=CLIP-G, Training-free=false2024.11 | 65.3 | 3.6 | |
| Chat-UniVi2025.01 | 65 | 3.6 | |
| Video-ChatGPTLLM Size=7B, Vision Encoder=CLIP-L, Training-free=false2024.11 | 64.9 | 3.3 | |
| Video-ChatGPT2025.01 | 64.9 | 3.3 | |
| VITA (8x7B)# Frames=1 fps (max 32)2024.10 | 63.9 | 3.7 | |
| Qwen2 VL (7B)# Frames=2 fps (max 768)2024.10 | 61.1 | 3.5 | |
| VideoChatLLM Size=7B, Vision Encoder=CLIP-G, Training-free=false2024.11 | 56.3 | 2.8 | |
| FrozenBiLMSupervision=100% (fully-supervised)2022.06 | 54.8 | — | |
| Video-LLaMALLM Size=7B, Vision Encoder=CLIP-G, Training-free=false2024.11 | 51.6 | 2.5 | |
| Video-LLaMA2025.01 | 51.6 | 2.5 | |
| FrozenBiLMSupervision=10% (few-shot)2022.06 | 51 | — | |
| FrozenBiLMSupervision=1% (few-shot)2022.06 | 46.5 | — | |
| FrozenBiLMTraining Data=WebVid10M, Speech=true, Zero-shot=true2022.06 | 33.8 | — | |
| FrozenBiLMSupervision=0% (zero-shot)2022.06 | 33.8 | — | |
| FrozenBiLMTraining Data=WebVid10M, Speech=false, Zero-shot=true2022.06 | 33.7 | — | |
| FrozenBiLMTraining Data=WebVid10M, Zero-shot evaluation=true2022.06 | 30 | — | |
| Just AskTraining Data=HowToVQA69M + WebVidVQA3M, Speech=false, Zero-shot=true2022.06 | 13.5 | — | |
| Just AskTraining Data=HowToVQA69M + WebVidVQA3M, Zero-shot evaluation=true2022.06 | 13.5 | — | |
| CLIP ViT-L/14Training Data=400M image-texts, Speech=false, Zero-shot=true2022.06 | 7.2 | — | |
| CLIP ViT-L/14Training Data=400M image-texts, Zero-shot evaluation=true2022.06 | 7.2 | — | |
| RandomZero-shot=true2022.06 | 0.1 | — | |
| RandomTraining Data=—, Zero-shot evaluation=true2022.06 | 0.1 | — |