List your company
ResearchFormerly WizWand

Video Question Answering on DeVE-QA (test)

71.8Accuracy (QA)DeVi-Gemini-2.0
39.97648.23856.564.762Sep 6, 2024
Updated 8mo ago

Evaluation results

MethodLinks
DeVi-Gemini-2.0LLM=Gemini-2.0, Evaluation protocol=zero-shot, Video Input=raw video2024.0971.8
DeVi-GPT-4oFrames=16, LLM=GPT-4o, Evaluation protocol=zero-shot2024.0971.2
LongVAFrames=48, LLM=Qwen2-Extended, Access Type=Open-source, Evaluation protocol=zero-shot2024.0964.9
VideoAgentLLM=GPT-4, Access Type=API-based, Evaluation protocol=zero-shot2024.0964.5
LLoViLLM=GPT-4, Access Type=API-based, Evaluation protocol=zero-shot2024.0963.8
PLLAVAFrames=16, LLM=LLaVA-Next-7B, Access Type=Open-source, Evaluation protocol=zero-shot2024.0963.7
Qwen2-VLFrames=32, LLM=Qwen2-7B, Access Type=Open-source, Evaluation protocol=zero-shot2024.0963.5
Gemini 2.0LLM=Gemini-2.0, Access Type=API-based, Evaluation protocol=zero-shot, Video Input=raw video2024.0963.4
GPT-4oFrames=16, LLM=GPT-4o, Access Type=API-based, Evaluation protocol=zero-shot2024.0962.6
LLaVA-OVFrames=16, LLM=Qwen2-7B, Access Type=Open-source, Evaluation protocol=zero-shot2024.0961.9
VideoLLaMA2Frames=16, LLM=Mistral-7B-Instruct, Access Type=Open-source, Evaluation protocol=zero-shot2024.0961.3
SeViLAFrames=32, LLM=BLIP-2, Access Type=Open-source, Evaluation protocol=zero-shot2024.0961.2
IG-VLMFrames=6, LLM=LLaVA-1.6-7B, Access Type=API-based, Evaluation protocol=zero-shot2024.0960.2
Videochat2Frames=16, LLM=Vicuna-7B, Access Type=Open-source, Evaluation protocol=zero-shot2024.0958.7
LLaMA-adapter (SFT)LLM=LLAMA-7B, Access Type=Open-source, Evaluation protocol=fine-tuned2024.0958.3
LLaVA-NeXT-VideoFrames=32, LLM=Vicuna-7B, Access Type=Open-source, Evaluation protocol=zero-shot2024.0957.1
Video-LLAVAFrames=8, LLM=Vicuna-7B, Access Type=Open-source, Evaluation protocol=zero-shot2024.0956.2
ViperGPTLLM=GPT-3, Access Type=API-based, Evaluation protocol=zero-shot2024.0955.1
VFCFrames=32, LLM=PaLM, Access Type=Open-source, Evaluation protocol=zero-shot2024.0949.5
InternVideoFrames=16, LLM=CLIP Text Encoder, Access Type=Open-source, Evaluation protocol=zero-shot2024.0948.3
VideoLLaMAFrames=8, LLM=Vicuna-7B, Access Type=Open-source, Evaluation protocol=zero-shot2024.0941.2