List your company
ResearchFormerly WizWand

Image Captioning on COCO (Semantic Consistency Score)

78.6Semantic Consistency ScoreClean
28.243241.316654.3967.4634Mar 29, 2026
Updated 7mo ago

Evaluation results

MethodLinks
CleanModel=LLaVA-1.5, Image Encoder=OpenAI CLIP ViT-L/14, Params=7B2026.0378.6
Natural Light AttackModel=LLaVA-1.5, Image Encoder=OpenAI CLIP ViT-L/14, Params=7B2026.0377
CleanModel=InstructBLIP (FlanT5XL), Image Encoder=EVA-CLIP ViT-G/14, Params=4.1B2026.0376.5
CleanModel=Blip-2 (FlanT5XL, ViT-L), Image Encoder=OpenAI CLIP ViT-L/14, Params=3.4B2026.0375.1
CleanModel=Blip-2 (FlanT5XL), Image Encoder=EVA-CLIP ViT-G/14, Params=4.1B2026.0374.96
Shadow AttackModel=LLaVA-1.5, Image Encoder=OpenAI CLIP ViT-L/14, Params=7B2026.0374.6
CleanModel=LLaVA-1.6, Image Encoder=OpenAI CLIP ViT-L/14, Params=7B2026.0372.1
Natural Light AttackModel=InstructBLIP (FlanT5XL), Image Encoder=EVA-CLIP ViT-G/14, Params=4.1B2026.0372.07
Natural Light AttackModel=LLaVA-1.6, Image Encoder=OpenAI CLIP ViT-L/14, Params=7B2026.0371.7
Natural Light AttackModel=Blip-2 (FlanT5XL), Image Encoder=EVA-CLIP ViT-G/14, Params=4.1B2026.0371.27
Shadow AttackModel=LLaVA-1.6, Image Encoder=OpenAI CLIP ViT-L/14, Params=7B2026.0371.17
Natural Light AttackModel=Blip-2 (FlanT5XL, ViT-L), Image Encoder=OpenAI CLIP ViT-L/14, Params=3.4B2026.0370.77
CleanModel=OpenFlamingo, Image Encoder=OpenAI CLIP ViT-L/14, Params=3B2026.0370.2
Shadow AttackModel=InstructBLIP (FlanT5XL), Image Encoder=EVA-CLIP ViT-G/14, Params=4.1B2026.0369.77
Natural Light AttackModel=OpenFlamingo, Image Encoder=OpenAI CLIP ViT-L/14, Params=3B2026.0369.53
Shadow AttackModel=Blip-2 (FlanT5XL), Image Encoder=EVA-CLIP ViT-G/14, Params=4.1B2026.0368.8
Shadow AttackModel=Blip-2 (FlanT5XL, ViT-L), Image Encoder=OpenAI CLIP ViT-L/14, Params=3.4B2026.0368.57
Shadow AttackModel=OpenFlamingo, Image Encoder=OpenAI CLIP ViT-L/14, Params=3B2026.0367.8
Wrinkle-Based Structural Attack (random)Model=InstructBLIP (FlanT5XL), Image Encoder=EVA-CLIP ViT-G/14, Params=4.1B2026.0363.92
ITAModel=LLaVA-1.5, Image Encoder=OpenAI CLIP ViT-L/14, Params=7B2026.0363.73
ITAModel=InstructBLIP (FlanT5XL), Image Encoder=EVA-CLIP ViT-G/14, Params=4.1B2026.0363.2
ITAModel=Blip-2 (FlanT5XL), Image Encoder=EVA-CLIP ViT-G/14, Params=4.1B2026.0362.01
ITAModel=LLaVA-1.6, Image Encoder=OpenAI CLIP ViT-L/14, Params=7B2026.0361.6
Wrinkle-Based Structural Attack (random)Model=Blip-2 (FlanT5XL, ViT-L), Image Encoder=OpenAI CLIP ViT-L/14, Params=3.4B2026.0361.01
ITAModel=Blip-2 (FlanT5XL, ViT-L), Image Encoder=OpenAI CLIP ViT-L/14, Params=3.4B2026.0360.93
Wrinkle-Based Structural Attack (random)Model=Blip-2 (FlanT5XL), Image Encoder=EVA-CLIP ViT-G/14, Params=4.1B2026.0359.48
Wrinkle-Based Structural Attack (best)Model=Blip-2 (FlanT5XL, ViT-L), Image Encoder=OpenAI CLIP ViT-L/14, Params=3.4B2026.0358.21
Wrinkle-Based Structural Attack (best)Model=Blip-2 (FlanT5XL), Image Encoder=EVA-CLIP ViT-G/14, Params=4.1B2026.0357.08
Wrinkle-Based Structural Attack (random)Model=LLaVA-1.5, Image Encoder=OpenAI CLIP ViT-L/14, Params=7B2026.0356.58
Wrinkle-Based Structural Attack (best)Model=InstructBLIP (FlanT5XL), Image Encoder=EVA-CLIP ViT-G/14, Params=4.1B2026.0356.57
Wrinkle-Based Structural Attack (best)Model=LLaVA-1.5, Image Encoder=OpenAI CLIP ViT-L/14, Params=7B2026.0354.51
Wrinkle-Based Structural Attack (random)Model=LLaVA-1.6, Image Encoder=OpenAI CLIP ViT-L/14, Params=7B2026.0354.45
ITAModel=OpenFlamingo, Image Encoder=OpenAI CLIP ViT-L/14, Params=3B2026.0353.93
Wrinkle-Based Structural Attack (best)Model=LLaVA-1.6, Image Encoder=OpenAI CLIP ViT-L/14, Params=7B2026.0353.15
Wrinkle-Based Structural Attack (random)Model=OpenFlamingo, Image Encoder=OpenAI CLIP ViT-L/14, Params=3B2026.0333.59
Wrinkle-Based Structural Attack (best)Model=OpenFlamingo, Image Encoder=OpenAI CLIP ViT-L/14, Params=3B2026.0330.18