List your company
ResearchFormerly WizWand

Image Captioning on Flickr30K (Karpathy split)

123.1CIDErInstructBLIP
27.31652.18377.05101.917May 27, 2022Aug 30, 2022Dec 4, 2022Mar 9, 2023Jun 13, 2023Sep 16, 2023Dec 21, 2023
Updated 8mo ago

Evaluation results

MethodLinks
InstructBLIPvisual encoder=EVA-g, glue layer=QFormer, LLM=Vicuna-7B, Res.=224, param=188M2023.12123.1
GITShot=full2022.0598.5
GIT2Shot=full2022.0598.5
GITShot=full2022.0598.5
CogVLMTrain Data=1.5B, Protocol=zero-shot2023.1194.9
GITLShot=full2022.0592.4
InternVL-Chatvisual encoder=IViT-6B, glue layer=QLLAMA, LLM=Vicuna-13B, Res.=336, param=13B2023.1292.2
InternVL-Chatvisual encoder=IViT-6B, glue layer=QLLAMA, LLM=Vicuna-13B, Res.=224, param=90M2023.1289.9
InternVL-Chatvisual encoder=IViT-6B, glue layer=QLLAMA, LLM=Vicuna-7B, Res.=224, param=64M2023.1289.7
GIT2Shot=290 (1%)2022.0588.2
ASMglue layer=QFormer, LLM decoder=Husky-7B, Zero-shot=true2023.1287.7
GITShot=290 (1%)2022.0586.6
GITShot=290 (1%)2022.0586.6
Qwen-VLzero-shot=true, backbone=Qwen-7B2023.0885.8
Qwen-VLglue layer=VL-Adapter, LLM decoder=Qwen-7B, Zero-shot=true2023.1285.8
Qwen-VLvisual encoder=CLIP-G, glue layer=VL-Adapter, LLM=Qwen-7B, Res.=448, param=9.6B2023.1285.8
Qwen-VLTrain Data=1.4B, Protocol=zero-shot2023.1185.8
InstructBLIP-FlanT5-XLmode=specialist2023.0884.5
InstructBLIPzero-shot=true, backbone=Vicuna-13B2023.0882.8
InstructBLIPglue layer=QFormer, LLM decoder=Vicuna-13B, Zero-shot=true2023.1282.8
InstructBLIPvisual encoder=EVA-g, glue layer=QFormer, LLM=Vicuna-13B, Res.=224, param=188M2023.1282.8
InstructBLIPTrain Data=129M, Protocol=zero-shot2023.1182.4
GIT2Shot=322022.0582
GITBShot=full2022.0581.8
Qwen-VL-Chatzero-shot=true2023.0881
Qwen-VL-Chatglue layer=VL-Adapter, LLM decoder=Qwen-7B, Zero-shot=true2023.1281
Qwen-VL-Chatvisual encoder=CLIP-G, glue layer=VL-Adapter, LLM=Qwen-7B, Res.=448, param=9.6B2023.1281
Kosmos-2zero-shot=true2023.0880.5
GITShot=322022.0580.5
GITShot=322022.0580.5
GIT2Shot=162022.0579.6
InternVL-Gglue layer=Cross-Attn, LLM decoder=QLLaMA, Zero-shot=true2023.1279.2
FlamingoShot=162022.0578.9
FlamingoShot=162022.0578.9
GITShot=162022.0578
GITShot=162022.0578
FlamingoShot=322022.0575.4
GITLShot=290 (1%)2022.0575.4
FlamingoShot=322022.0575.4
Shikrazero-shot=true, backbone=Vicuna-13B2023.0873.9
Shikra-13Bglue layer=Linear, LLM decoder=Vicuna-13B, Zero-shot=true2023.1273.9
Shikravisual encoder=CLIP-L, glue layer=Linear, LLM=Vicuna-13B, Res.=224, param=7B2023.1273.9
GITBShot=290 (1%)2022.0571.8
BLIP-2zero-shot=true, backbone=Vicuna-13B2023.0871.6
BLIP-2glue layer=QFormer, LLM decoder=Vicuna-13B, Zero-shot=true2023.1271.6
BLIP-2visual encoder=EVA-g, glue layer=QFormer, LLM=Vicuna-13B, Res.=224, param=188M2023.1271.6
Zhou et al. (2020)Shot=full2022.0568.5
GITLShot=322022.0568.5
Zhou et al. (2020)Shot=full2022.0568.5
Flamingo-80Bzero-shot=true2023.0867.2
Flamingo-80Bglue layer=Cross-Attn, LLM decoder=Chinchilla-70B, Zero-shot=true2023.1267.2
FlamingoTrain Data=2.3B, Protocol=zero-shot2023.1167.2
FlamingoShot=02022.0567.2
FlamingoShot=02022.0567.2
Kosmos-1zero-shot=true2023.0867.1
KOSMOS-2Inference mode=Zero-shot, Instruction tuning=false, Beam size=5, Resolution=224x2242023.0666.7
KOSMOS-2glue layer=Linear, LLM decoder=KOSMOS-1, Zero-shot=true2023.1266.7
GITBShot=322022.0566.4
GITBShot=162022.0565.8
IDEFICS-80B-Ivisual encoder=CLIP-H, glue layer=Cross-Attn, LLM=LLaMA-65B, Res.=224, param=15B2023.1265.3
KOSMOS-1Inference mode=Zero-shot, Instruction tuning=false, Beam size=5, Resolution=224x2242023.0665.2
GITLShot=162022.0564.4
Flamingo-9BInference mode=Zero-shot, Instruction tuning=false, Beam size=5, Resolution=224x2242023.0661.5
Flamingo-9Bzero-shot=true2023.0861.5
Flamingo-9Bglue layer=Cross-Attn, LLM decoder=Chinchilla-7B, Zero-shot=true2023.1261.5
Flamingo-3BInference mode=Zero-shot, Instruction tuning=false, Beam size=5, Resolution=224x2242023.0660.6
IDEFICS-80Bvisual encoder=CLIP-H, glue layer=Cross-Attn, LLM=LLaMA-65B, Res.=224, param=15B2023.1253.7
GIT2Train Data=12.9B, Protocol=zero-shot2023.1150.7
GIT2Shot=02022.0550.7
GITTrain Data=0.8B, Protocol=zero-shot2023.1149.6
GITShot=02022.0549.6
GITShot=02022.0549.6
METALMInference mode=Zero-shot, Instruction tuning=false, Beam size=5, Resolution=224x2242023.0643.4
GITLShot=02022.0539.2
GITBShot=02022.0535.2
FewVLMInference mode=Zero-shot, Instruction tuning=false, Beam size=5, Resolution=224x2242023.0631