Video Question Answering on DeVE-QA (test)
71.8Accuracy (QA)DeVi-Gemini-2.0
Evaluation Results
| Method | Links | |
|---|---|---|
| DeVi-Gemini-2.0LLM=Gemini-2.0, Evaluation protocol=zero-shot, Video Input=raw video2024.09 | 71.8 | |
| DeVi-GPT-4oFrames=16, LLM=GPT-4o, Evaluation protocol=zero-shot2024.09 | 71.2 | |
| LongVAFrames=48, LLM=Qwen2-Extended, Access Type=Open-source, Evaluation protocol=zero-shot2024.09 | 64.9 | |
| VideoAgentLLM=GPT-4, Access Type=API-based, Evaluation protocol=zero-shot2024.09 | 64.5 | |
| LLoViLLM=GPT-4, Access Type=API-based, Evaluation protocol=zero-shot2024.09 | 63.8 | |
| PLLAVAFrames=16, LLM=LLaVA-Next-7B, Access Type=Open-source, Evaluation protocol=zero-shot2024.09 | 63.7 | |
| Qwen2-VLFrames=32, LLM=Qwen2-7B, Access Type=Open-source, Evaluation protocol=zero-shot2024.09 | 63.5 | |
| Gemini 2.0LLM=Gemini-2.0, Access Type=API-based, Evaluation protocol=zero-shot, Video Input=raw video2024.09 | 63.4 | |
| GPT-4oFrames=16, LLM=GPT-4o, Access Type=API-based, Evaluation protocol=zero-shot2024.09 | 62.6 | |
| LLaVA-OVFrames=16, LLM=Qwen2-7B, Access Type=Open-source, Evaluation protocol=zero-shot2024.09 | 61.9 | |
| VideoLLaMA2Frames=16, LLM=Mistral-7B-Instruct, Access Type=Open-source, Evaluation protocol=zero-shot2024.09 | 61.3 | |
| SeViLAFrames=32, LLM=BLIP-2, Access Type=Open-source, Evaluation protocol=zero-shot2024.09 | 61.2 | |
| IG-VLMFrames=6, LLM=LLaVA-1.6-7B, Access Type=API-based, Evaluation protocol=zero-shot2024.09 | 60.2 | |
| Videochat2Frames=16, LLM=Vicuna-7B, Access Type=Open-source, Evaluation protocol=zero-shot2024.09 | 58.7 | |
| LLaMA-adapter (SFT)LLM=LLAMA-7B, Access Type=Open-source, Evaluation protocol=fine-tuned2024.09 | 58.3 | |
| LLaVA-NeXT-VideoFrames=32, LLM=Vicuna-7B, Access Type=Open-source, Evaluation protocol=zero-shot2024.09 | 57.1 | |
| Video-LLAVAFrames=8, LLM=Vicuna-7B, Access Type=Open-source, Evaluation protocol=zero-shot2024.09 | 56.2 | |
| ViperGPTLLM=GPT-3, Access Type=API-based, Evaluation protocol=zero-shot2024.09 | 55.1 | |
| VFCFrames=32, LLM=PaLM, Access Type=Open-source, Evaluation protocol=zero-shot2024.09 | 49.5 | |
| InternVideoFrames=16, LLM=CLIP Text Encoder, Access Type=Open-source, Evaluation protocol=zero-shot2024.09 | 48.3 | |
| VideoLLaMAFrames=8, LLM=Vicuna-7B, Access Type=Open-source, Evaluation protocol=zero-shot2024.09 | 41.2 |