Compositional Reasoning on SugarCrepe++
75.2Average PerformanceCLIPS (ViT-L/14)
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| CLIPS (ViT-L/14)Backbone=ViT-L/142026.03 | 75.2 | 76.6 | 66.6 | 59.4 | 44.3 | — | — | — | — | — | |
| C^2LIPBackbone=ViT-B/162026.03 | 75 | 79.7 | 75.3 | 55.2 | 44.2 | — | — | — | — | — | |
| SigLIP2-ViT-B/16Backbone=ViT-B/162026.03 | 74.4 | 77.7 | 69.7 | 57.9 | 42 | — | — | — | — | — | |
| SigLIP ViT-L/16-res256Backbone=ViT-L/16, Input Resolution=256x2562026.03 | 73.4 | 75.4 | 64.9 | 55.6 | 42.1 | — | — | — | — | — | |
| BLIP-BBackbone=ViT-B2026.03 | 71.3 | 76.7 | 86 | 52.1 | 41.3 | — | — | — | — | — | |
| CLIP-A (ViT-L/14)Backbone=ViT-L/142026.03 | 70.5 | 74.7 | 74.8 | 45.8 | 30.8 | — | — | — | — | — | |
| FLAVABackbone=ViT-B2026.03 | 69.9 | 74.3 | 75 | 56.6 | 51 | — | — | — | — | — | |
| NegCLIP2025.09 | 63.7 | — | — | — | — | — | — | — | — | — | |
| NegCLIPModel=NegCLIP2025.09 | 63.7 | — | — | — | — | 57.5 | 52 | 92.1 | 73 | 43.9 | |
| LSS2025.09 | 61.8 | — | — | — | — | — | — | — | — | — | |
| LSSModel=LSS2025.09 | 61.8 | — | — | — | — | 52.2 | 53.4 | 91.3 | 74.9 | 36.5 | |
| SigLIP2025.09 | 57.5 | — | — | — | — | — | — | — | — | — | |
| SigLIPModel=SigLIP2025.09 | 57.5 | — | — | — | — | 51.1 | 49.8 | 85.2 | 69.9 | 31.4 | |
| CE-CLIP2025.09 | 56.3 | — | — | — | — | — | — | — | — | — | |
| CE-CLIPModel=CE-CLIP2025.09 | 56.3 | — | — | — | — | 41.6 | 52.6 | 85.7 | 68.3 | 33.3 | |
| LongCLIP2025.09 | 54.7 | — | — | — | — | — | — | — | — | — | |
| LongCLIPModel=LongCLIP2025.09 | 54.7 | — | — | — | — | 40.8 | 48.4 | 89.1 | 65.6 | 29.6 | |
| DreamLIP2025.09 | 54.1 | — | — | — | — | — | — | — | — | — | |
| DreamLIPModel=DreamLIP2025.09 | 54.1 | — | — | — | — | 53 | 45.1 | 81.2 | 58.2 | 32.9 | |
| CLIP2025.09 | 53.3 | — | — | — | — | — | — | — | — | — | |
| CLIPModel=CLIP2025.09 | 53.3 | — | — | — | — | 39.1 | 47.4 | 85.3 | 62.4 | 32.5 | |
| DCIP12025.09 | 51.3 | — | — | — | — | — | — | — | — | — | |
| DCIP1Model=DCIP12025.09 | 51.3 | — | — | — | — | 41.8 | 39.5 | 80.7 | 56.6 | 38 | |
| DACLLM2025.09 | 44 | — | — | — | — | — | — | — | — | — | |
| DACLLMModel=DACLLM2025.09 | 44 | — | — | — | — | 29.5 | 48.6 | 70.1 | 50 | 19.6 |