Image Captioning on COCO (Semantic Consistency Score)
78.6Semantic Consistency ScoreClean
Evaluation Results
| Method | Links | |
|---|---|---|
| CleanModel=LLaVA-1.5, Image Encoder=OpenAI CLIP ViT-L/14, Params=7B2026.03 | 78.6 | |
| Natural Light AttackModel=LLaVA-1.5, Image Encoder=OpenAI CLIP ViT-L/14, Params=7B2026.03 | 77 | |
| CleanModel=InstructBLIP (FlanT5XL), Image Encoder=EVA-CLIP ViT-G/14, Params=4.1B2026.03 | 76.5 | |
| CleanModel=Blip-2 (FlanT5XL, ViT-L), Image Encoder=OpenAI CLIP ViT-L/14, Params=3.4B2026.03 | 75.1 | |
| CleanModel=Blip-2 (FlanT5XL), Image Encoder=EVA-CLIP ViT-G/14, Params=4.1B2026.03 | 74.96 | |
| Shadow AttackModel=LLaVA-1.5, Image Encoder=OpenAI CLIP ViT-L/14, Params=7B2026.03 | 74.6 | |
| CleanModel=LLaVA-1.6, Image Encoder=OpenAI CLIP ViT-L/14, Params=7B2026.03 | 72.1 | |
| Natural Light AttackModel=InstructBLIP (FlanT5XL), Image Encoder=EVA-CLIP ViT-G/14, Params=4.1B2026.03 | 72.07 | |
| Natural Light AttackModel=LLaVA-1.6, Image Encoder=OpenAI CLIP ViT-L/14, Params=7B2026.03 | 71.7 | |
| Natural Light AttackModel=Blip-2 (FlanT5XL), Image Encoder=EVA-CLIP ViT-G/14, Params=4.1B2026.03 | 71.27 | |
| Shadow AttackModel=LLaVA-1.6, Image Encoder=OpenAI CLIP ViT-L/14, Params=7B2026.03 | 71.17 | |
| Natural Light AttackModel=Blip-2 (FlanT5XL, ViT-L), Image Encoder=OpenAI CLIP ViT-L/14, Params=3.4B2026.03 | 70.77 | |
| CleanModel=OpenFlamingo, Image Encoder=OpenAI CLIP ViT-L/14, Params=3B2026.03 | 70.2 | |
| Shadow AttackModel=InstructBLIP (FlanT5XL), Image Encoder=EVA-CLIP ViT-G/14, Params=4.1B2026.03 | 69.77 | |
| Natural Light AttackModel=OpenFlamingo, Image Encoder=OpenAI CLIP ViT-L/14, Params=3B2026.03 | 69.53 | |
| Shadow AttackModel=Blip-2 (FlanT5XL), Image Encoder=EVA-CLIP ViT-G/14, Params=4.1B2026.03 | 68.8 | |
| Shadow AttackModel=Blip-2 (FlanT5XL, ViT-L), Image Encoder=OpenAI CLIP ViT-L/14, Params=3.4B2026.03 | 68.57 | |
| Shadow AttackModel=OpenFlamingo, Image Encoder=OpenAI CLIP ViT-L/14, Params=3B2026.03 | 67.8 | |
| Wrinkle-Based Structural Attack (random)Model=InstructBLIP (FlanT5XL), Image Encoder=EVA-CLIP ViT-G/14, Params=4.1B2026.03 | 63.92 | |
| ITAModel=LLaVA-1.5, Image Encoder=OpenAI CLIP ViT-L/14, Params=7B2026.03 | 63.73 | |
| ITAModel=InstructBLIP (FlanT5XL), Image Encoder=EVA-CLIP ViT-G/14, Params=4.1B2026.03 | 63.2 | |
| ITAModel=Blip-2 (FlanT5XL), Image Encoder=EVA-CLIP ViT-G/14, Params=4.1B2026.03 | 62.01 | |
| ITAModel=LLaVA-1.6, Image Encoder=OpenAI CLIP ViT-L/14, Params=7B2026.03 | 61.6 | |
| Wrinkle-Based Structural Attack (random)Model=Blip-2 (FlanT5XL, ViT-L), Image Encoder=OpenAI CLIP ViT-L/14, Params=3.4B2026.03 | 61.01 | |
| ITAModel=Blip-2 (FlanT5XL, ViT-L), Image Encoder=OpenAI CLIP ViT-L/14, Params=3.4B2026.03 | 60.93 | |
| Wrinkle-Based Structural Attack (random)Model=Blip-2 (FlanT5XL), Image Encoder=EVA-CLIP ViT-G/14, Params=4.1B2026.03 | 59.48 | |
| Wrinkle-Based Structural Attack (best)Model=Blip-2 (FlanT5XL, ViT-L), Image Encoder=OpenAI CLIP ViT-L/14, Params=3.4B2026.03 | 58.21 | |
| Wrinkle-Based Structural Attack (best)Model=Blip-2 (FlanT5XL), Image Encoder=EVA-CLIP ViT-G/14, Params=4.1B2026.03 | 57.08 | |
| Wrinkle-Based Structural Attack (random)Model=LLaVA-1.5, Image Encoder=OpenAI CLIP ViT-L/14, Params=7B2026.03 | 56.58 | |
| Wrinkle-Based Structural Attack (best)Model=InstructBLIP (FlanT5XL), Image Encoder=EVA-CLIP ViT-G/14, Params=4.1B2026.03 | 56.57 | |
| Wrinkle-Based Structural Attack (best)Model=LLaVA-1.5, Image Encoder=OpenAI CLIP ViT-L/14, Params=7B2026.03 | 54.51 | |
| Wrinkle-Based Structural Attack (random)Model=LLaVA-1.6, Image Encoder=OpenAI CLIP ViT-L/14, Params=7B2026.03 | 54.45 | |
| ITAModel=OpenFlamingo, Image Encoder=OpenAI CLIP ViT-L/14, Params=3B2026.03 | 53.93 | |
| Wrinkle-Based Structural Attack (best)Model=LLaVA-1.6, Image Encoder=OpenAI CLIP ViT-L/14, Params=7B2026.03 | 53.15 | |
| Wrinkle-Based Structural Attack (random)Model=OpenFlamingo, Image Encoder=OpenAI CLIP ViT-L/14, Params=3B2026.03 | 33.59 | |
| Wrinkle-Based Structural Attack (best)Model=OpenFlamingo, Image Encoder=OpenAI CLIP ViT-L/14, Params=3B2026.03 | 30.18 |