Compositional Vision-Language Reasoning on Winoground
89.5Text ScoreMTurk Human
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| MTurk Human2023.05 | 89.5 | 88.5 | 85.5 | |
| MTurk Human2023.11 | 89.5 | 88.5 | 85.5 | |
| MTurk Human2023.09 | 89.5 | 88.5 | 85.5 | |
| MTurk Human2026.03 | 89.5 | 88.5 | 85.5 | |
| Qwen3+SG (mt)Strategy=mt (multi-turn)2026.03 | 76 | 72 | 66 | |
| GPT-4VPrompt=Chain-of-Thought, Strategy=Description then Decision2023.11 | 75.25 | 68.75 | 58.75 | |
| Qwen3-VL-Think (plain)Strategy=plain2026.03 | 74.5 | 72 | 62.8 | |
| Qwen3-VL-Think + SGStrategy=+ SG2026.03 | 74.5 | 73.5 | 64.2 | |
| GPT-4VPrompt=Standard2023.11 | 69.25 | 46.25 | 39.25 | |
| GPT4-V2023.09 | 69.25 | 46.25 | 39.25 | |
| SCRAMBLe-Molmo-7BParameters=7B2026.03 | 66.8 | 66.3 | 54.8 | |
| Molmo-7BParameters=7B2026.03 | 62.8 | 61.8 | 49.5 | |
| CECE (LLaVA-1.5+1.6)Model Architecture=LLaVA-1.5+1.62026.03 | 55 | 61.3 | 47.5 | |
| LLaVA-1.5-13BModel Architecture=1.5-13B2026.03 | 51.5 | 50.5 | 36.5 | |
| FiberType=CLIP-based encoding similarity2023.11 | 51.49 | 31.49 | 27.5 | |
| VQ2Type=Vision Large Language Model2023.11 | 47 | 42.2 | 30.5 | |
| VQ22023.09 | 47 | 42.2 | 30.5 | |
| PALIType=Vision Large Language Model2023.11 | 46.5 | 38 | 28.75 | |
| PALI2023.09 | 46.5 | 38 | 28.75 | |
| MMICLType=Vision Large Language Model2023.11 | 45.5 | 44.99 | 43 | |
| MMICLBackbone=FLAN-T5-XXL2023.09 | 45 | 45 | 43 | |
| METERType=CLIP-based encoding similarity2023.11 | 44.99 | 22.75 | 18.75 | |
| Blip-22023.09 | 44 | 26 | 23.5 | |
| IAIS LargeConfidence Measure=IAIS2023.05 | 42.5 | 19.75 | 16 | |
| VladVABackbone=LLAVA-1.5-7B2024.12 | 40.5 | 17.5 | 12.8 | |
| CACR BaseConfidence Measure=CACR2023.05 | 39.25 | 17.75 | 14.25 | |
| UNITER LargeConfidence Measure=ITM2023.05 | 38 | 14 | 10.5 | |
| VinVLConfidence Measure=ITM2023.05 | 37.75 | 17.75 | 14.5 | |
| EVA-CLIPNumber of parameters=8B2024.12 | 36.5 | 14.8 | 10.3 | |
| EVA-CLIPNumber of parameters=18B2024.12 | 35.8 | 15 | 10.5 | |
| OpenCLIPBackbone=ViT-BigG/142024.12 | 35.5 | 15.5 | 12 | |
| BLIP (ITM Large)Model Architecture=ITM Large2026.03 | 35 | 22 | 22 | |
| EVA-02-CLIPBackbone=ViT-E/14+2024.12 | 33.8 | 14 | 10.8 | |
| SigLIP2025.09 | 32.8 | 12.8 | 10.3 | |
| E5-VBackbone=LLAVA-Next-8B2024.12 | 32.3 | 14.8 | 11.3 | |
| OpenCLIPBackbone=ViT-G/142024.12 | 32 | 12.8 | 9.3 | |
| E5-VBackbone=LLAVA-1.5-7B2024.12 | 31.3 | 17.4 | 10.5 | |
| CLIPstatus=baseline2023.03 | 31.25 | 10.5 | 8 | |
| CLIP (ViT-B/32)Backbone=ViT-B/322026.03 | 30.8 | 10.5 | 8 | |
| LSSBackbone=ViT-B/162025.09 | 30.8 | 11.3 | 8.8 | |
| OFA LargeConfidence Measure=ITM2023.05 | 30.75 | 10.25 | 7.25 | |
| CLIPType=CLIP-based encoding similarity2023.11 | 30.75 | 10.5 | 8 | |
| BLIPBackbone=ViT-L2024.12 | 30.5 | 10 | 7.8 | |
| syn-CyCLIPTraining Data=LAION + SyViC, Fine-tuned=true2023.03 | 30 | 10.75 | 8.25 | |
| syn-CLIPfinetuned_on=SyViC2023.03 | 30 | 11.5 | 9.5 | |
| BLIP2Backbone=ViT-L2024.12 | 29.5 | 10.5 | 8.5 | |
| NegCLIP2024.12 | 29.5 | 10.5 | 8 | |
| OFA LargeConfidence Measure=TLC-A2023.05 | 29.25 | 27 | 17.5 | |
| CyCLIPTraining Data=LAION2023.03 | 28.5 | 9.5 | 7.25 | |
| DreamLIP2025.09 | 28.3 | 15 | 10.8 | |
| CLIPBackbone=ViT-L2024.12 | 27.5 | 12.3 | 8.3 | |
| OFA BaseConfidence Measure=ITM2023.05 | 26.75 | 10.75 | 6.5 | |
| Random Chance2023.05 | 25 | 25 | 16.67 | |
| Random Chance2023.11 | 25 | 25 | 16.67 | |
| CLIPBackbone=ViT-B2024.12 | 25 | 10.5 | 7.3 | |
| Random Chance2026.03 | 25 | 25 | 6.3 | |
| OFA BaseConfidence Measure=TLC-A2023.05 | 24.5 | 23.5 | 13.75 | |
| OFA TinyConfidence Measure=ITM2023.05 | 22.75 | 7.75 | 4.5 | |
| TIFAType=Vision Large Language Model2023.11 | 19 | 12.5 | 11.3 | |
| LLaVA-1.5-7B2024.12 | 18.5 | 11.3 | 6.5 | |
| OFA TinyConfidence Measure=TLC-A2023.05 | 16.5 | 15.75 | 6.75 |