Video Question Answering on How2QA
93.2AccT+T (+ASR)
Evaluation Results
| Method | Links | |
|---|---|---|
| T+T (+ASR)Input=dense/1fps, Speech Input=true2023.05 | 93.2 | |
| T+TInput=dense/1fps, Speech Input=true2023.05 | 92.4 | |
| FrozenBiLMNumber of Frames=10, Speech Input=true2023.05 | 86.7 | |
| JustAskNumber of Frames=20, Speech Input=true2023.05 | 84.4 | |
| JustAskzero-shot=false2023.12 | 84.4 | |
| ViLAFrames Numbers=42023.12 | 83.9 | |
| SEVILA†Number of Frames=32 -> 4, Keyframe Selection=zero-shot LOCALIZER2023.05 | 83.7 | |
| SEVILANumber of Frames=32 -> 4, Keyframe Selection=refined LOCALIZER2023.05 | 83.6 | |
| SeViLAFrames Numbers=42023.12 | 83.6 | |
| SeViLAzero-shot=false2023.12 | 83.6 | |
| ViLAFrames Numbers=22023.12 | 82.8 | |
| BLIP-2 concat (ANSWERER)Number of Frames=4, Protocol=feature concatenation2023.05 | 82.2 | |
| BLIP-2Frames Numbers=42023.12 | 82.2 | |
| FrozenBiLMNumber of Frames=102023.05 | 81.5 | |
| FrozenBiLMFrames Numbers=102023.12 | 81.5 | |
| BLIP-2 votingNumber of Frames=4, Protocol=majority voting2023.05 | 79.6 | |
| InternVideoNumber of Frames=8, Tested_by_authors=true2023.05 | 79 | |
| InternVideoFrames Numbers=82023.12 | 79 | |
| LF-VILA2022.10 | 76.1 | |
| ResNet-SF2022.10 | 74.3 | |
| HERO2022.10 | 74.3 | |
| HEROInput=dense/1fps, Speech Input=true2023.05 | 73.8 | |
| CLIP-SF2022.10 | 72.9 | |
| SeViLA†Number of Frames=32 -> 4, Zero-shot=true2023.05 | 72.3 | |
| SeViLaevaluation_mode=Zero-shot2024.02 | 72.3 | |
| Q-ViDevaluation_mode=Zero-shot2024.02 | 71.4 | |
| BLIP-2concat (Answerer)Number of Frames=4, Zero-shot=true2023.05 | 70.8 | |
| BLIP-2concatevaluation_mode=Zero-shot2024.02 | 70.8 | |
| BLIPv2 ViTG FlanT5xxlzero-shot=true, vision encoder=ViTG, LLM=FlanT5xxl, number of frames=42023.09 | 69.8 | |
| BLIP-2votingNumber of Frames=4, Zero-shot=true2023.05 | 69.8 | |
| BLIP-2votingevaluation_mode=Zero-shot2024.02 | 69.8 | |
| CLIPimplementation_source=[28]2022.10 | 69.3 | |
| AnyMAL-Image 70Bzero-shot=true, vision encoder=ViT-G, LLM size=70B, number of frames=42023.09 | 68.1 | |
| OneLLM-7Bzero-shot=true2023.12 | 65.7 | |
| Internvideozero-shot=true, number of frames=82023.09 | 62.2 | |
| InternVideo*Number of Frames=8, Zero-shot=true2023.05 | 62.2 | |
| InternVideoevaluation_mode=Zero-shot2024.02 | 62.2 | |
| InternVideozero-shot=true2023.12 | 62.2 | |
| AnyMAL-Video 70Bzero-shot=true, vision encoder=Internvideo, LLM size=70B, number of frames=82023.09 | 60 | |
| AnyMAL-Image 13Bzero-shot=true, vision encoder=ViT-G, LLM size=13B, number of frames=42023.09 | 59.6 | |
| AnyMAL-13Bzero-shot=true2023.12 | 59.6 | |
| FrozenBiLMNumber of Frames=10, speech input or dense frames=true, Zero-shot=true2023.05 | 58.4 | |
| FrozenBiLMzero-shot=true2023.12 | 58.4 | |
| AnyMAL-Video 13Bzero-shot=true, vision encoder=Internvideo, LLM size=13B, number of frames=82023.09 | 54.8 | |
| JustAskNumber of Frames=20, speech input or dense frames=true, Zero-shot=true2023.05 | 51.1 | |
| FrozenBiLMNumber of Frames=10, Zero-shot=true2023.05 | 41.9 | |
| FrozenBILMevaluation_mode=Zero-shot2024.02 | 41.9 |