List your company
ResearchFormerly WizWand

Visual Question Answering on OKVQA (val)

66.1VQA ScorePaLM-E
0.78817.74434.751.656Jun 25, 2021Feb 21, 2022Oct 20, 2022Jun 18, 2023Feb 14, 2024Oct 12, 2024Jun 11, 2025
Updated 8mo ago

Evaluation results

MethodLinks
PaLM-EParameters=562B, Fine-tuning=per-task, OCR_pipeline=without2023.0566.1
PaLI-XParameters=55B, Fine-tuning=per-task, OCR_pipeline=without2023.0566.1
PaLM-ENumber of parameters=562B2023.0566.1
PaLI-XNumber of parameters=55B2023.0566.1
PaLM-EParameters=562B2023.1066.1
PaLI-XParameters=55B2023.1066.1
PalmE-562Btrained_on_QA_datasets=true2024.0366.1
Emu2-ChatLLM=LLaMA-33B, Trained during SFT stage=true2023.1164.8
CogVLM-ChatLLM=Vicuna-7B, Trained during SFT stage=true2023.1164.8
multi-modal exploration-exploitation reinforcement learning frameworkBase Model=Qwen-VL, Selection Strategy=Reinforcement Learning2025.0664.7
PaLIParameters=17B, Fine-tuning=per-task, OCR_pipeline=without2023.0564.5
PaLINumber of parameters=17B2023.0564.5
PaLI-17BParameters=17B2023.1064.5
PaLI-17Btrained_on_QA_datasets=true2024.0364.5
SPHINX-2kLLM=LLaMA2 13B, Trained during SFT stage=true2023.1162.6
randomBase Model=Qwen-VL, Selection Strategy=random2025.0661.1
similarityBase Model=Qwen-VL, Selection Strategy=Similarity2025.0660.6
De-DiffusionVisual Encoder=ViT-L, LLM=PaLM 2-L, Trainable Parameters=135M, Shots=322023.1160.6
FuyuLLM=Fuyu-8B, Trained during SFT stage=true2023.1160.6
BM25Base Model=Qwen-VL, Selection Strategy=BM252025.0660.3
PaLI-3Parameters=5B2023.1060.1
De-DiffusionVisual Encoder=ViT-L, LLM=PaLM 2-L, Trainable Parameters=135M, Shots=42023.1158.2
FlamingoParameters=80B, Fine-tuning=per-task, OCR_pipeline=without2023.0557.8
FlamingoNumber of parameters=80B2023.0557.8
FlamingoParameters=80B, Shots=322023.1057.8
IDEFICS-80BLLM=Llama65B, Trainable Parameters=14B, Shots=322023.1157.8
Flamingo-80BLLM=Chinchilla70B, Trainable Parameters=10B, Shots=322023.1157.8
mPLUG-Owl2LLM=LLaMA2-7B, Trained during SFT stage=true2023.1157.7
zero-shotBase Model=Qwen-VL, Selection Strategy=zero-shot2025.0657.5
Flamingo-80BLLM=Chinchilla70B, Trainable Parameters=10B, Shots=42023.1157.4
De-DiffusionVisual Encoder=ViT-L, LLM=PaLM 2-L, Trainable Parameters=135M, Shots=02023.1157
Qwen-VL-ChatLLM=Qwen-7B, Trained during SFT stage=true2023.1156.6
Unified-IO2LLM=UIO-2XXL, Trained during SFT stage=true2023.1155.5
PalmE-12Btrained_on_QA_datasets=true2024.0355.5
Idefics2-baseSize=8B, Architecture=fully autoregressive, # tokens per image=64, Few-shot=8 random in-context examples, Setting=open-ended2024.0554.6
De-DiffusionVisual Encoder=ViT-L, LLM=PaLM 2-S, Trainable Parameters=135M, Shots=42023.1153.5
De-DiffusionVisual Encoder=ViT-L, LLM=PaLM 2-S, Trainable Parameters=135M, Shots=322023.1153.3
IDEFICS-80BLLM=Llama65B, Trainable Parameters=14B, Shots=42023.1152.4
PaLI-3Btrained_on_QA_datasets=true2024.0352.4
FlexCap-LLMzero-shot=true2024.0352.1
ViperGPTzero-shot=true2024.0351.9
De-DiffusionVisual Encoder=ViT-L, LLM=PaLM 2-S, Trainable Parameters=135M, Shots=02023.1151.4
MM1Size=7B, Architecture=fully autoregressive, # tokens per image=144, Few-shot=8 random in-context examples, Setting=open-ended2024.0551.4
Flamingo-9BLLM=Chinchilla7B, Trainable Parameters=2B, Shots=322023.1151
Flamingo-80BLLM=Chinchilla70B, Trainable Parameters=10B, Shots=02023.1150.6
Flamingozero-shot=true2024.0350.6
FlamingoSize=9B, Architecture=cross-attention, Few-shot=8 random in-context examples, Setting=open-ended2024.0550
Flamingo-9BLLM=Chinchilla7B, Trainable Parameters=2B, Shots=42023.1149.3
PICa-FullLLM=GPT-3, Trainable Parameters=0, Shots=162023.1148
Idefics1Size=9B, Architecture=cross-attention, Few-shot=8 random in-context examples, Setting=open-ended2024.0547.7
BLIP-2Visual Encoder=ViT-g, LLM=FlanT5XXL, Trainable Parameters=108M, Shots=0, In-domain COCO training=true2023.1145.9
BLIPv2zero-shot=true2024.0345.9
IDEFICS-80BLLM=Llama65B, Trainable Parameters=14B, Shots=02023.1145.2
Flamingo-9BLLM=Chinchilla7B, Trainable Parameters=2B, Shots=02023.1144.7
multi-modal exploration-exploitation reinforcement learning frameworkBase Model=LLaVA, Selection Strategy=Reinforcement Learning2025.0644.3
DreamLLMLLM=Vicuna-7B, Trained during SFT stage=false2023.1144.3
LENSLLM=FlanT5XXL, Trainable Parameters=0, Shots=02023.1143.3
AnyMALVisual Encoder=VIT-G, LLM=Llama270B, Trainable Parameters=0, Shots=0, In-domain COCO training=true2023.1142.6
OpenFlamingo-9BLLM=MPT7B, Shots=322023.1142.4
OpenFlamingoSize=9B, Architecture=cross-attention, Few-shot=8 random in-context examples, Setting=open-ended2024.0541.1
OpenFlamingo-9BLLM=MPT7B, Shots=42023.1140.1
MAVEXuses_task_training_data=true2021.0639.4
OpenFlamingoLLM=MPT-7B, Trained during SFT stage=false2023.1138.3
OpenFlamingo-9BLLM=MPT7B, Shots=02023.1137.8
IDEFICS-InstructLLM=LLaMA-65B, Trained during SFT stage=false2023.1136.9
VL-HyperPELTNumber of Samples=20002022.0335.86
VL-HyperPELTNumber of Samples=10002022.0335.72
VL-HyperPELTNumber of Samples=5002022.0335.56
VL-HyperPELTNumber of Samples=1002022.0334.99
VL-AdapterNumber of Samples=20002022.0334.87
VL-HyperPELTNumber of Samples=322022.0334.86
VL-HyperPELTNumber of Samples=162022.0334.72
CLIP-T5Number of Samples=20002022.0334.62
CLIP-T5Number of Samples=10002022.0334.59
VL-AdapterNumber of Samples=10002022.0334.57
CLIP-T5Number of Samples=5002022.0334.43
CLIP-T5Number of Samples=1002022.0334.27
CLIP-T5Number of Samples=322022.0333.87
CLIP-T5Number of Samples=162022.0333.68
VL-AdapterNumber of Samples=5002022.0333.35
VL-HyperPELTNumber of Samples=42022.0333.25
VL-AdapterNumber of Samples=1002022.0333.03
CLIP-T5Number of Samples=42022.0332.65
VL-AdapterNumber of Samples=322022.0332.07
VL-AdapterNumber of Samples=162022.0331.86
VL-AdapterNumber of Samples=42022.0331.83
similarityBase Model=LLaVA, Selection Strategy=Similarity2025.0622.7
Frozen VQAn-shot=0, uses_task_training_data=false2021.0619.6
Frozenn-shot=4, uses_task_training_data=false2021.0612.6
Frozen VQA-blindn-shot=0, uses_task_training_data=false2021.0612.5
Frozenn-shot=1, uses_task_training_data=false2021.069.7
Frozen train-blindn-shot=1, uses_task_training_data=false2021.067.2
Frozen 400mLMn-shot=4, uses_task_training_data=false2021.066.6
Frozenn-shot=0, uses_task_training_data=false2021.065.9
Frozen 400mLMn-shot=1, uses_task_training_data=false2021.065.9
Frozen finetunedn-shot=4, uses_task_training_data=false2021.064.6
Frozen finetunedn-shot=0, uses_task_training_data=false2021.064.2
Frozen finetunedn-shot=1, uses_task_training_data=false2021.064.1
Frozen 400mLMn-shot=0, uses_task_training_data=false2021.064
Frozen train-blindn-shot=0, uses_task_training_data=false2021.063.3