Video Question Answering on How2QA (test)
93.2AccuracyText Tokens + Text Transformer
Evaluation results
| Method | Links | |
|---|---|---|
| Text Tokens + Text TransformerFv, Ft=CLIP [26], Extra MM Samples=0, Delta GPU hours=0, ASR=true2022.06 | 93.2 | |
| Text Tokens + Text TransformerFv, Ft=CLIP [26], Extra MM Samples=0, Delta GPU hours=0, ASR=false2022.06 | 92.4 | |
| FrozenBiLMFv, Ft=CLIP [26], Extra MM Samples=10M, Delta GPU hours=160, ASR=true2022.06 | 86.7 | |
| VQA-TFv, Ft=S3D [21], Extra MM Samples=69M + 3M, Delta GPU hours=350 + 30, ASR=false2022.06 | 85.3 | |
| Continuous Features + Multimodal TransformerFv, Ft=S3D [21], Extra MM Samples=69M, Delta GPU hours=400, ASR=true2022.06 | 84.8 | |
| Text Tokens + Text TransformerFv, Ft=S3D [21], Extra MM Samples=0, Delta GPU hours=0, ASR=true2022.06 | 84.6 | |
| Continuous Features + Multimodal TransformerFv, Ft=S3D [21], Extra MM Samples=69M, Delta GPU hours=400, ASR=false2022.06 | 84.4 | |
| SiaSamReaExtra MM Samples=5.6M + 80K, ASR=false2022.06 | 84.1 | |
| Text Tokens + Text TransformerFv, Ft=S3D [21], Extra MM Samples=0, Delta GPU hours=0, ASR=false2022.06 | 82.9 | |
| FrozenBiLMFv, Ft=CLIP [26], Extra MM Samples=10M, Delta GPU hours=160, ASR=false2022.06 | 81.5 | |
| HEROInput Modality=video+subtitle2020.05 | 73.81 | |
| ATPFeatures=CLIP ViT-B-322022.06 | 65.1 | |
| HERO+Features=CLIP, Preprocessing=Same as ATP2022.06 | 60.9 | |
| HEROFeatures=CLIP2022.06 | 60.4 | |
| CLIPTemporal Aggregation=mean pooling2022.06 | 55.7 | |
| VQA-TPretraining Data=HowToVQA69M, Zero-shot=true2020.12 | 51.1 | |
| CLIPTemporal Aggregation=single-frame2022.06 | 50.1 | |
| VQA-TPretraining Data=HowTo100M, Zero-shot=true2020.12 | 46.2 | |
| QA-TPretraining Data=HowToVQA69M, Zero-shot=true2020.12 | 38.4 | |
| Random2022.06 | 25 | |
| RandomPretraining Data=None, Zero-shot=true2020.12 | 25 |