Vision-Language Compositional Reasoning on Winoground 1.0 (test)
89.5Text ScoreHuman
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| HumanSource=taken from original paper2026.05 | 89.5 | 88.5 | 85.5 | |
| VQAScoreMethod Category=LLM-based, Backbone=CLIP-FlanT5-XXL2026.05 | 60.8 | 55.3 | 44.8 | |
| BEiTScoreMethod Category=Encoder-based, Backbone=L/142026.05 | 55.5 | 43.3 | 34.5 | |
| BEiTScoreMethod Category=Encoder-based, Backbone=L/14, Stage=without Stage12026.05 | 48 | 35.3 | 27 | |
| IAISMethod category=Embedding-based2024.01 | 42.5 | 19.5 | 16 | |
| FLEURMethod Category=LLM-based2026.05 | 40 | 45.3 | 28.8 | |
| EXPERTMethod Category=LLM-based2026.05 | 40 | 46 | 28.3 | |
| BEiTScoreMethod Category=Encoder-based, Backbone=B/322026.05 | 39.5 | 28.5 | 18.3 | |
| CACRMethod category=Embedding-based2024.01 | 39.3 | 17.8 | 14.3 | |
| VinVLMethod Category=Encoder-based, Source=taken from original paper2026.05 | 37.8 | 17.8 | 14.5 | |
| CLIP-SMethod Category=Encoder-based, Backbone=ViT-B2026.05 | 35.5 | 11.3 | 8.5 | |
| BEiTScoreMethod Category=Encoder-based, Backbone=B/32, Stage=without Stage12026.05 | 33.3 | 22.5 | 14.3 | |
| FLAVAMethod Category=Encoder-based, Variant=ITM, Source=taken from original paper2026.05 | 32.3 | 20.5 | 14.3 | |
| PAC-S++Method Category=Encoder-based, Backbone=ViT-B2026.05 | 31.8 | 8.8 | 6.5 | |
| PAC-S++Method Category=Encoder-based, Backbone=ViT-L2026.05 | 31.5 | 15 | 9.8 | |
| LongCLIP-SMethod Category=Encoder-based, Backbone=ViT-B2026.05 | 31 | 9 | 7.3 | |
| CLIPMethod category=Embedding-based2024.01 | 30.8 | 10.5 | 8 | |
| CLIP-SMethod Category=Encoder-based, Backbone=ViT-L2026.05 | 30.8 | 11.5 | 9.5 | |
| LongCLIP-SMethod Category=Encoder-based, Backbone=ViT-L2026.05 | 29.8 | 11.5 | 9.3 | |
| PAC-SMethod Category=Encoder-based, Backbone=ViT-B2026.05 | 27 | 10 | 6.8 | |
| PAC-SMethod Category=Encoder-based, Backbone=ViT-L2026.05 | 26 | 11 | 7.8 | |
| Random ChanceSource=taken from original paper2026.05 | 25 | 25 | 16.7 | |
| SPECSMethod Category=Encoder-based2026.05 | 9.3 | 7.8 | 3 |