Video Question Answering on IVQA
53.8AccuracyInstructBLIP
Evaluation results
| Method | Links | ||
|---|---|---|---|
| InstructBLIPzero-shot=true, backbone=FlanT5-XXL2023.05 | 53.8 | — | |
| InstructBLIPzero-shot=true, backbone=FlanT5-XL2023.05 | 53.1 | — | |
| InstructBLIPzero-shot=true, backbone=Vicuna-7B2023.05 | 52.2 | — | |
| MMICLbackbone=Instruct-FLAN-T5-XXL2023.09 | 51.83 | — | |
| InstructBLIPzero-shot=true, backbone=Vicuna-13B2023.05 | 51 | — | |
| BLIP-2backbone=FLANT5-XXL2023.09 | 49.38 | — | |
| MMICLbackbone=Instruct-FLAN-T5-XL2023.09 | 49.28 | — | |
| BLIP-2zero-shot=true, backbone=FlanT5-XXL2023.05 | 45.8 | — | |
| Flamingo-80Bshots=4-Shot2023.09 | 44.1 | — | |
| MMICLbackbone=FLAN-T5-XL2023.09 | 41.68 | — | |
| MMICLbackbone=FLAN-T5-XXL2023.09 | 41.13 | — | |
| Flamingozero-shot=true, backbone=80B2023.05 | 40.7 | — | |
| Flamingo-80Bshots=Zero-Shot2023.09 | 40.7 | — | |
| BLIP-2zero-shot=true, backbone=FlanT5-XL2023.05 | 40.4 | — | |
| CoTokenizationPre-training dataset=HowTo100M, GFLOPs=672022.08 | 38.2 | — | |
| Flamingo-9Bshots=4-Shot2023.09 | 37.7 | — | |
| BLIP-2backbone=FLANT5-XL2023.09 | 37.3 | — | |
| InstructBLIPbackbone=FLANT5-XXL2023.09 | 36.15 | — | |
| VQA-TPre-training dataset=HowToVQA69M2022.08 | 35.4 | — | |
| Flamingozero-shot=true, backbone=9B2023.05 | 35.2 | — | |
| Flamingo-3Bshots=4-Shot2023.09 | 35.2 | — | |
| Flamingo-9Bshots=Zero-Shot2023.09 | 35.2 | — | |
| FrozenBiLM (Ours)Pretraining Data=WebVid10M, Finetuning Data=VQA2022.06 | 34.6 | — | |
| Flamingozero-shot=true, backbone=3B2023.05 | 32.7 | — | |
| Flamingo-3Bshots=Zero-Shot2023.09 | 32.7 | — | |
| R2A2023.09 | 29.3 | — | |
| VQA-TPre-training dataset=HowTo100M2022.08 | 28.1 | — | |
| BLIP-2zero-shot=true, backbone=Vicuna-7B2023.05 | 27.5 | — | |
| FrozenBiLM (no image-VQA training)Pretraining Data=WebVid10M, Finetuning Data=∅2022.06 | 26.8 | — | |
| InstructBLIPbackbone=FLANT5-XL2023.09 | 25.18 | — | |
| BLIP-2zero-shot=true, backbone=Vicuna-13B2023.05 | 23.5 | — | |
| FrozenBiLM (no cross-modal training)Pretraining Data=∅, Finetuning Data=VQA2022.06 | 14.6 | — | |
| FrozenBiLMEvaluation Protocol=finetuning2022.12 | — | 39.6 | |
| JustAskEvaluation Protocol=finetuning2022.12 | — | 35.4 | |
| VideoCoCaEvaluation Protocol=finetuning2022.12 | — | 39 |