List your company
ResearchFormerly WizWand

Video Question Answering on iVQA (test)

60.9AccuracyMoReVQA
-2.33214.08430.546.916Jun 5, 2022Sep 25, 2022Jan 15, 2023May 8, 2023Aug 28, 2023Dec 18, 2023Apr 9, 2024
Updated 8mo ago

Evaluation results

MethodLinks
MoReVQAFT=false2024.0460.9——
JCEFFT=false2024.0456.9——
InstructBLIPFT=false, Variant=FlanT5XXL2024.0453.8——
InstructBLIPFT=false, Variant=FlanT5XL2024.0451.1——
ProViQEvaluation protocol=zero-shot2023.1250.7——
BLIP-2FT=false, Variant=FlanT5XXL2024.0445.8——
Supervised SOTAEvaluation protocol=supervised2023.1240.9——
Flamingo-80BAdaptation type=Training based Adaptation, Language model=Chinchilla-like, Language model #params=80B, Vision model=NFNet-F6, Vision model #params=629M, Evaluation protocol=Zero-shot2023.0640.7——
Text Tokens + Text TransformerFv, Ft=CLIP [26], Extra MM Samples=0, Delta GPU hours=0, ASR=true2022.0640.2——
Text+TextFT=false2024.0440.2——
FrozenBiLMFv, Ft=CLIP [26], Extra MM Samples=10M, Delta GPU hours=160, ASR=false2022.0639.7——
FrozenBiLMFT=true2024.0439.7——
FrozenBiLMFv, Ft=CLIP [26], Extra MM Samples=10M, Delta GPU hours=160, ASR=true2022.0639.6——
VideoCoCaFT=false2024.0439——
Text Tokens + Text TransformerFv, Ft=CLIP [26], Extra MM Samples=0, Delta GPU hours=0, ASR=false2022.0636.9——
Text Tokens + Text TransformerFv, Ft=S3D [21], Extra MM Samples=0, Delta GPU hours=0, ASR=true2022.0636.8——
Continuous Features + Multimodal TransformerFv, Ft=S3D [21], Extra MM Samples=69M, Delta GPU hours=400, ASR=true2022.0636——
Continuous Features + Multimodal TransformerFv, Ft=S3D [21], Extra MM Samples=69M, Delta GPU hours=400, ASR=false2022.0635.4——
VQA-TFv, Ft=S3D [21], Extra MM Samples=69M + 3M, Delta GPU hours=350 + 30, ASR=false2022.0635.2——
Flamingo-9BAdaptation type=Training based Adaptation, Language model=Chinchilla-like, Language model #params=8.7B, Vision model=NFNet-F6, Vision model #params=629M, Evaluation protocol=Zero-shot2023.0635.2——
Flamingo-3BAdaptation type=Training based Adaptation, Language model=Chinchilla-like, Language model #params=2.6B, Vision model=NFNet-F6, Vision model #params=629M, Evaluation protocol=Zero-shot2023.0632.7——
Text Tokens + Text TransformerFv, Ft=S3D [21], Extra MM Samples=0, Delta GPU hours=0, ASR=false2022.0631.6——
R2AAdaptation type=Language based Adaptation, Language model=DeBERTa-v2-XL, Language model #params=890M, Vision model=ViT-L/14, Vision model #params=300M, Evaluation protocol=Zero-shot2023.0629.3——
FrozenBiLMFT=false2024.0427.3——
FrozenBiLMEvaluation protocol=zero-shot2023.1226.8——
FrozenBiLMAdaptation type=Training based Adaptation, Language model=DeBERTa-v2-XL, Language model #params=890M, Vision model=ViT-L/14, Vision model #params=300M, Evaluation protocol=Zero-shot2023.0626.2——
Just AskAdaptation type=Training based Adaptation, Language model=DistilBERT, Language model #params=66M, Vision model=S3D, Vision model #params=12M, Evaluation protocol=Zero-shot2023.0613.3——
Just AskEvaluation protocol=zero-shot2023.1213.3——
CLIP VIT-L/14Adaptation type=Training based Adaptation, Language model=Custom, Language model #params=123M, Vision model=ViT-L/14, Vision model #params=300M, Evaluation protocol=Zero-shot2023.069.2——
CLIP-VIT-L/14Evaluation protocol=zero-shot2023.129.2——
RandomEvaluation protocol=zero-shot2023.120.1——
QA-TPretraining Data=HowToVQA69M, Zero-shot=true2020.12—4.423.2
RandomPretraining Data=None, Zero-shot=true2020.12—0.090.9
VQA-TPretraining Data=HowTo100M, Zero-shot=true2020.12—1.911.9
VQA-TPretraining Data=HowToVQA69M, Zero-shot=true2020.12—12.243.3