Visual Question Answering on OK-VQA v1.1 (test)
66.1VQA ScorePaLM-E
Evaluation results
| Method | Links | ||
|---|---|---|---|
| PaLM-EBase Model=PaLM-E (562B), Knowledge Source=PaLM-E2023.09 | 66.1 | — | |
| PALI (17B)Base Model=PALI (17B), Knowledge Source=PALI2023.09 | 64.5 | — | |
| RA-VQA-v2 (BLIP 2)Base Model=BLIP 2 (T5-XL), K=5, Knowledge Source=GoogleSearch2023.09 | 62.08 | 62.01 | |
| ProphetBase Model=GPT-3, Knowledge Source=GPT-32023.09 | 61.11 | — | |
| RA-VQA-v2 (BLIP 2) w/o ROIBase Model=BLIP 2 (T5-XL), K=5, Knowledge Source=GoogleSearch2023.09 | 60.75 | 61.63 | |
| RA-VQA-v2 (BLIP 2) w/o ROI & VEBase Model=BLIP 2 (T5-XL), K=5, Knowledge Source=GoogleSearch2023.09 | 60.41 | 59.95 | |
| PromptCapBase Model=GPT-3, Knowledge Source=GPT-32023.09 | 60.4 | — | |
| RA-VQA-v2 (BLIP 2) w/o ROI & Late-interactionBase Model=BLIP 2 (T5-XL), K=5, Knowledge Source=GoogleSearch, Retriever=DPR2023.09 | 59.21 | 60.18 | |
| RA-VQA-v2 (BLIP 2) w/o ROI & VE & Late-interactionBase Model=BLIP 2 (T5-XL), K=5, Knowledge Source=GoogleSearch, Retriever=DPR2023.09 | 58.7 | 59.49 | |
| RA-VQA-v2 (BLIP 2) w/o Late-interactionBase Model=BLIP 2 (T5-XL), K=5, Knowledge Source=GoogleSearch, Retriever=DPR2023.09 | 58.2 | 59 | |
| REVIVEBase Model=GPT-3, Knowledge Source=Wikipedia + GPT-32023.09 | 58 | — | |
| FlamingoBase Model=Flamingo (80B), Knowledge Source=Flamingo2023.09 | 57.8 | — | |
| PALI (15B)Base Model=PALI (15B), Knowledge Source=PALI2023.09 | 56.5 | — | |
| BLIP 2 (fine-tuned) w/o knowledgeBase Model=BLIP 2 (T5-XL)2023.09 | 55.44 | 59.49 | |
| RA-VQA-v2 (T5-large)Base Model=T5-large, K=5, Knowledge Source=GoogleSearch2023.09 | 54.85 | 58.85 | |
| RA-VQA (joint training)Base Model=T5-large, K=50, Knowledge Source=GoogleSearch2023.09 | 54.48 | 59.41 | |
| KAT-EnsembleBase Model=T5-large, GPT-3, K=40, Knowledge Source=Wikipedia + GPT-32023.09 | 54.41 | — | |
| PALI (3B)Base Model=PALI (3B), Knowledge Source=PALI2023.09 | 52.4 | — | |
| RA-VQA-v2 (T5-large) w/o ROI & VE & Late-interactionBase Model=T5-large, K=5, Knowledge Source=GoogleSearch, Retriever=DPR2023.09 | 51.45 | 55.89 | |
| RA-VQABase Model=T5-large, K=5, Knowledge Source=GoogleSearch2023.09 | 51.22 | 55.77 | |
| TRiG EnsembleBase Model=T5-large, K=100, Knowledge Source=Wikipedia2023.09 | 50.5 | 54.73 | |
| PICaBase Model=GPT-3, Knowledge Source=GPT-32023.09 | 48 | — | |
| T5-large (fine-tuned) w/o knowledgeBase Model=T5-large2023.09 | 47.52 | 51.38 | |
| VRRK=100, Knowledge Source=GoogleSearch2023.09 | 45.08 | — | |
| KAT-T5Base Model=T5-large, K=40, Knowledge Source=Wikipedia2023.09 | 44.25 | — | |
| MAVEXKnowledge Source=Wikipedia + ConceptNet + Google Images2023.09 | 39.4 | — | |
| KRISPKnowledge Source=ConceptNet + Wikipedia2023.09 | 38.35 | — | |
| ConceptBERTKnowledge Source=ConceptNet2023.09 | 33.66 | — |