Visual Question Answering on COCO
68ScoreClean (Baseline)
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Clean (Baseline)Image Encoder=OpenAI CLIP ViT-L/14, Model=LLaVA-1.5, Params=7B2026.03 | 68 | — | — | — | — | — | |
| Natural Light AttackImage Encoder=OpenAI CLIP ViT-L/14, Model=LLaVA-1.5, Params=7B2026.03 | 68 | 0 | — | — | — | — | |
| Clean (Baseline)Image Encoder=EVA-CLIP ViT-G/14, Model=InstructBLIP (FlanT5XL), Params=4.1B2026.03 | 68 | — | — | — | — | — | |
| LLaVA-1.5Image Encoder=OpenAI CLIP ViT-L/14, Params=7B2026.03 | 68 | — | 68 | 67 | 48 | 47.45 | |
| InstructBLIP(FlanT5XL)Image Encoder=EVA-CLIP ViT-G/14, Params=4.1B2026.03 | 68 | — | 64 | 62 | 44 | 43.67 | |
| CleanImage Encoder=OpenAI CLIP ViT-L/14, Model=LLaVA-1.5, #Params=7B2026.04 | 68 | — | — | — | — | — | |
| Natural Light AttackImage Encoder=OpenAI CLIP ViT-L/14, Model=LLaVA-1.5, #Params=7B2026.04 | 68 | — | — | — | — | — | |
| CleanImage Encoder=EVA-CLIP ViT-G/14, Model=InstructBLIP (FlanT5XL), #Params=4.1B2026.04 | 68 | — | — | — | — | — | |
| LLaVA-1.5Backbone=LLaVA-1.5-7B2026.03 | 67.45 | — | — | — | — | — | |
| Shadow AttackImage Encoder=OpenAI CLIP ViT-L/14, Model=LLaVA-1.5, Params=7B2026.03 | 67 | 1 | — | — | — | — | |
| Shadow AttackImage Encoder=OpenAI CLIP ViT-L/14, Model=LLaVA-1.5, #Params=7B2026.04 | 67 | 1 | — | — | — | — | |
| CoVFTBackbone=LLaVA-1.5-7B2026.03 | 66.96 | — | — | — | — | — | |
| LLaVA-1.5Backbone=LLaVA-1.5-13B2026.03 | 66.82 | — | — | — | — | — | |
| Full fine-tuningBackbone=LLaVA-1.5-7B2026.03 | 65.71 | — | — | — | — | — | |
| SVPTBackbone=LLaVA-1.5-7B2026.03 | 65.34 | — | — | — | — | — | |
| BitFitBackbone=LLaVA-1.5-7B2026.03 | 65.34 | — | — | — | — | — | |
| CoVFTBackbone=LLaVA-1.5-13B2026.03 | 65.22 | — | — | — | — | — | |
| VPTBackbone=LLaVA-1.5-7B2026.03 | 65.09 | — | — | — | — | — | |
| FreezeBackbone=LLaVA-1.5-13B2026.03 | 65.09 | — | — | — | — | — | |
| LoRABackbone=LLaVA-1.5-7B2026.03 | 64.22 | — | — | — | — | — | |
| Full fine-tuningBackbone=LLaVA-1.5-13B2026.03 | 64.22 | — | — | — | — | — | |
| Clean (Baseline)Image Encoder=OpenAI CLIP ViT-L/14, Model=LLaVA-1.6, Params=7B2026.03 | 64 | — | — | — | — | — | |
| Shadow AttackImage Encoder=OpenAI CLIP ViT-L/14, Model=LLaVA-1.6, Params=7B2026.03 | 64 | 0 | — | — | — | — | |
| Natural Light AttackImage Encoder=EVA-CLIP ViT-G/14, Model=InstructBLIP (FlanT5XL), Params=4.1B2026.03 | 64 | 4 | — | — | — | — | |
| LLaVA-1.6Image Encoder=OpenAI CLIP ViT-L/14, Params=7B2026.03 | 64 | — | 63 | 64 | 43 | 48.87 | |
| CleanImage Encoder=OpenAI CLIP ViT-L/14, Model=LLaVA-1.6, #Params=7B2026.04 | 64 | — | — | — | — | — | |
| Shadow AttackImage Encoder=OpenAI CLIP ViT-L/14, Model=LLaVA-1.6, #Params=7B2026.04 | 64 | — | — | — | — | — | |
| Natural Light AttackImage Encoder=EVA-CLIP ViT-G/14, Model=InstructBLIP (FlanT5XL), #Params=4.1B2026.04 | 64 | 4 | — | — | — | — | |
| FreezeBackbone=LLaVA-1.5-7B2026.03 | 63.73 | — | — | — | — | — | |
| BitFitBackbone=LLaVA-1.5-13B2026.03 | 63.22 | — | — | — | — | — | |
| Natural Light AttackImage Encoder=OpenAI CLIP ViT-L/14, Model=LLaVA-1.6, Params=7B2026.03 | 63 | 1 | — | — | — | — | |
| Clean (Baseline)Image Encoder=OpenAI CLIP ViT-L/14, Model=Blip-2 (FlanT5XL, ViT-L), Params=3.4B2026.03 | 63 | — | — | — | — | — | |
| BLIP-2(FlanT5XL ViT-L)Image Encoder=EVA-CLIP ViT-G/14, Params=3.4B2026.03 | 63 | — | 58 | 50 | 38 | 18.62 | |
| Natural Light AttackImage Encoder=OpenAI CLIP ViT-L/14, Model=LLaVA-1.6, #Params=7B2026.04 | 63 | 1 | — | — | — | — | |
| CleanImage Encoder=OpenAI CLIP ViT-L/14, Model=Blip-2 (FlanT5XL ViT-L), #Params=3.4B2026.04 | 63 | — | — | — | — | — | |
| Shadow AttackImage Encoder=EVA-CLIP ViT-G/14, Model=InstructBLIP (FlanT5XL), Params=4.1B2026.03 | 62 | 6 | — | — | — | — | |
| Shadow AttackImage Encoder=EVA-CLIP ViT-G/14, Model=InstructBLIP (FlanT5XL), #Params=4.1B2026.04 | 62 | 6 | — | — | — | — | |
| Natural Light AttackImage Encoder=OpenAI CLIP ViT-L/14, Model=Blip-2 (FlanT5XL, ViT-L), Params=3.4B2026.03 | 58 | 5 | — | — | — | — | |
| Natural Light AttackImage Encoder=OpenAI CLIP ViT-L/14, Model=Blip-2 (FlanT5XL ViT-L), #Params=3.4B2026.04 | 58 | 5 | — | — | — | — | |
| Wrinkle-Based Structural Attack (random)Image Encoder=OpenAI CLIP ViT-L/14, Model=LLaVA-1.5, Params=7B2026.03 | 55 | 13 | — | — | — | — | |
| Clean (Baseline)Image Encoder=EVA-CLIP ViT-G/14, Model=Blip-2 (FlanT5XL), Params=4.1B2026.03 | 54 | — | — | — | — | — | |
| Shadow AttackImage Encoder=EVA-CLIP ViT-G/14, Model=Blip-2 (FlanT5XL), Params=4.1B2026.03 | 54 | 0 | — | — | — | — | |
| BLIP-2(FlanT5XL)Image Encoder=EVA-CLIP ViT-G/14, Params=4.1B2026.03 | 54 | — | 53 | 54 | 33 | 19.72 | |
| CleanImage Encoder=EVA-CLIP ViT-G/14, Model=Blip-2 (FlanT5XL), #Params=4.1B2026.04 | 54 | — | — | — | — | — | |
| Shadow AttackImage Encoder=EVA-CLIP ViT-G/14, Model=Blip-2 (FlanT5XL), #Params=4.1B2026.04 | 54 | — | — | — | — | — | |
| Natural Light AttackImage Encoder=EVA-CLIP ViT-G/14, Model=Blip-2 (FlanT5XL), Params=4.1B2026.03 | 53 | 1 | — | — | — | — | |
| Natural Light AttackImage Encoder=EVA-CLIP ViT-G/14, Model=Blip-2 (FlanT5XL), #Params=4.1B2026.04 | 53 | 1 | — | — | — | — | |
| Shadow AttackImage Encoder=OpenAI CLIP ViT-L/14, Model=Blip-2 (FlanT5XL, ViT-L), Params=3.4B2026.03 | 50 | 13 | — | — | — | — | |
| Shadow AttackImage Encoder=OpenAI CLIP ViT-L/14, Model=Blip-2 (FlanT5XL ViT-L), #Params=3.4B2026.04 | 50 | 13 | — | — | — | — | |
| ITAImage Encoder=OpenAI CLIP ViT-L/14, Model=LLaVA-1.5, Params=7B2026.03 | 48 | 20 | — | — | — | — | |
| ITAImage Encoder=OpenAI CLIP ViT-L/14, Model=LLaVA-1.5, #Params=7B2026.04 | 48 | 20 | — | — | — | — | |
| Wrinkle-Based Structural Attack (random)Image Encoder=OpenAI CLIP ViT-L/14, Model=LLaVA-1.6, Params=7B2026.03 | 47 | 17 | — | — | — | — | |
| Wrinkle-Based Structural Attack (best)Image Encoder=OpenAI CLIP ViT-L/14, Model=LLaVA-1.5, Params=7B2026.03 | 45 | 23 | — | — | — | — | |
| Clean (Baseline)Image Encoder=OpenAI CLIP ViT-L/14, Model=OpenFlamingo, Params=3B2026.03 | 45 | — | — | — | — | — | |
| Wrinkle-Based Structural Attack (random)Image Encoder=EVA-CLIP ViT-G/14, Model=InstructBLIP (FlanT5XL), Params=4.1B2026.03 | 45 | 23 | — | — | — | — | |
| OpenFlamingoImage Encoder=EVA-CLIP ViT-G/14, Params=3B2026.03 | 45 | — | 39 | 44 | 19 | 14.29 | |
| CleanImage Encoder=OpenAI CLIP ViT-L/14, Model=OpenFlamingo, #Params=3B2026.04 | 45 | — | — | — | — | — | |
| Shadow AttackImage Encoder=OpenAI CLIP ViT-L/14, Model=OpenFlamingo, Params=3B2026.03 | 44 | 1 | — | — | — | — | |
| ITAImage Encoder=EVA-CLIP ViT-G/14, Model=InstructBLIP (FlanT5XL), Params=4.1B2026.03 | 44 | 24 | — | — | — | — | |
| Shadow AttackImage Encoder=OpenAI CLIP ViT-L/14, Model=OpenFlamingo, #Params=3B2026.04 | 44 | 1 | — | — | — | — | |
| ITAImage Encoder=EVA-CLIP ViT-G/14, Model=InstructBLIP (FlanT5XL), #Params=4.1B2026.04 | 44 | 24 | — | — | — | — | |
| ITAImage Encoder=OpenAI CLIP ViT-L/14, Model=LLaVA-1.6, Params=7B2026.03 | 43 | 21 | — | — | — | — | |
| ITAImage Encoder=OpenAI CLIP ViT-L/14, Model=LLaVA-1.6, #Params=7B2026.04 | 43 | 21 | — | — | — | — | |
| Wrinkle-Based Structural Attack (best)Image Encoder=EVA-CLIP ViT-G/14, Model=InstructBLIP (FlanT5XL), Params=4.1B2026.03 | 41 | 27 | — | — | — | — | |
| Wrinkle-Based Structural Attack (best)Image Encoder=OpenAI CLIP ViT-L/14, Model=LLaVA-1.6, Params=7B2026.03 | 40 | 24 | — | — | — | — | |
| Natural Light AttackImage Encoder=OpenAI CLIP ViT-L/14, Model=OpenFlamingo, Params=3B2026.03 | 39 | 6 | — | — | — | — | |
| Natural Light AttackImage Encoder=OpenAI CLIP ViT-L/14, Model=OpenFlamingo, #Params=3B2026.04 | 39 | 6 | — | — | — | — | |
| ITAImage Encoder=OpenAI CLIP ViT-L/14, Model=Blip-2 (FlanT5XL, ViT-L), Params=3.4B2026.03 | 38 | 25 | — | — | — | — | |
| ITAImage Encoder=OpenAI CLIP ViT-L/14, Model=Blip-2 (FlanT5XL ViT-L), #Params=3.4B2026.04 | 38 | 25 | — | — | — | — | |
| Wrinkle-Based Structural Attack (random)Image Encoder=OpenAI CLIP ViT-L/14, Model=Blip-2 (FlanT5XL, ViT-L), Params=3.4B2026.03 | 36 | 27 | — | — | — | — | |
| (2)Adaptation strategy=Model parameter adaptation2025.10 | 35.9 | — | — | — | — | — | |
| ITAImage Encoder=EVA-CLIP ViT-G/14, Model=Blip-2 (FlanT5XL), Params=4.1B2026.03 | 33 | 21 | — | — | — | — | |
| ITAImage Encoder=EVA-CLIP ViT-G/14, Model=Blip-2 (FlanT5XL), #Params=4.1B2026.04 | 33 | 21 | — | — | — | — | |
| Wrinkle-Based Structural Attack (best)Image Encoder=OpenAI CLIP ViT-L/14, Model=Blip-2 (FlanT5XL, ViT-L), Params=3.4B2026.03 | 32 | 31 | — | — | — | — | |
| Wrinkle-Based Structural Attack (random)Image Encoder=EVA-CLIP ViT-G/14, Model=Blip-2 (FlanT5XL), Params=4.1B2026.03 | 32 | 22 | — | — | — | — | |
| Wrinkle-Based Structural Attack (best)Image Encoder=EVA-CLIP ViT-G/14, Model=Blip-2 (FlanT5XL), Params=4.1B2026.03 | 28 | 26 | — | — | — | — | |
| InternVL3-2BModel Size=2B2026.02 | 27.9 | — | — | — | — | — | |
| Robobrain-2.0-3BModel Size=3B2026.02 | 27.2 | — | — | — | — | — | |
| InternVL3-8BModel Size=8B2026.02 | 26.5 | — | — | — | — | — | |
| MSLAImage Encoder=OpenAI CLIP ViT-L/14, Model=LLaVA-1.6, #Params=7B2026.04 | 26 | 38 | — | — | — | — | |
| Robobrain-2.0-7BModel Size=7B2026.02 | 25.2 | — | — | — | — | — | |
| MSLAImage Encoder=OpenAI CLIP ViT-L/14, Model=LLaVA-1.5, #Params=7B2026.04 | 24 | 44 | — | — | — | — | |
| InternVL3-1BModel Size=1B2026.02 | 23.7 | — | — | — | — | — | |
| Wrinkle-Based Structural Attack (random)Image Encoder=OpenAI CLIP ViT-L/14, Model=OpenFlamingo, Params=3B2026.03 | 20 | 25 | — | — | — | — | |
| ITAImage Encoder=OpenAI CLIP ViT-L/14, Model=OpenFlamingo, Params=3B2026.03 | 19 | 26 | — | — | — | — | |
| ITAImage Encoder=OpenAI CLIP ViT-L/14, Model=OpenFlamingo, #Params=3B2026.04 | 19 | 26 | — | — | — | — | |
| TTAugAdaptation strategy=Test-time Augmentation2025.10 | 16.9 | — | — | — | — | — | |
| Method ④test-time scaling=Other method 42025.10 | 16.7 | — | — | — | — | — | |
| (1)Adaptation strategy=(1)2025.10 | 16.4 | — | — | — | — | — | |
| RoboInter-Qwen-7BModel Size=7B, Backbone=Qwen2026.02 | 15.9 | — | — | — | — | — | |
| TTAugtest-time scaling=Method 52025.10 | 15.9 | — | — | — | — | — | |
| RoboInter-LLaVAOV-7BModel Size=7B, Backbone=LLaVA-OV2026.02 | 15.8 | — | — | — | — | — | |
| QwenVL2.5-3BModel Size=3B2026.02 | 15.7 | — | — | — | — | — | |
| RoboInter-Qwen-3BModel Size=3B, Backbone=Qwen2026.02 | 15.6 | — | — | — | — | — | |
| QwenVL2.5-7BModel Size=7B2026.02 | 15 | — | — | — | — | — | |
| MSLAImage Encoder=EVA-CLIP ViT-G/14, Model=InstructBLIP (FlanT5XL), #Params=4.1B2026.04 | 15 | 53 | — | — | — | — | |
| Wrinkle-Based Structural Attack (best)Image Encoder=OpenAI CLIP ViT-L/14, Model=OpenFlamingo, Params=3B2026.03 | 14 | 31 | — | — | — | — | |
| MSLAImage Encoder=EVA-CLIP ViT-G/14, Model=Blip-2 (FlanT5XL), #Params=4.1B2026.04 | 10 | 44 | — | — | — | — | |
| Baselinetest-time scaling=none2025.10 | 9.1 | — | — | — | — | — | |
| Baseline2025.10 | 9.1 | — | — | — | — | — |