Image-Text Matching on Winoground
89.5Text Agreement ScoreMTurk Human
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| MTurk Human2024.01 | 89.5 | 88.5 | 85.5 | — | |
| MMICL + CoCoTPrompting strategy=CoCoT2024.01 | 64.25 | 52.5 | 50.75 | — | |
| GPT-4V + CoCoTPrompting strategy=CoCoT2024.01 | 58.5 | 49.5 | 44.5 | — | |
| OPENFLAMINGO + CoCoTPrompting strategy=CoCoT2024.01 | 58.25 | 55.25 | 41.5 | — | |
| GPT-4VPrompting strategy=Standard2024.01 | 54.5 | 42.5 | 37.75 | — | |
| FIBER + EQSIMSetting=Fine-tuning with EQSIM regularization2023.03 | 51.49 | 31.49 | 27.5 | — | |
| FIBERSetting=Standard fine-tuning (FT) on Flickr30K2023.03 | 51.24 | 26.49 | 23 | — | |
| MMICL + CCoTPrompting strategy=CCoT2024.01 | 51 | 48 | 47.5 | — | |
| OPENFLAMINGO + DDCoTPrompting strategy=DDCoT2024.01 | 47.5 | 47.25 | 39 | — | |
| MMICL + DDCoTPrompting strategy=DDCoT2024.01 | 46.75 | 45 | 36.75 | — | |
| MMICLPrompting strategy=Standard2024.01 | 46.5 | 40.75 | 37.75 | — | |
| FIBERSetting=Direct evaluation after pre-training2023.03 | 46.25 | 25.75 | 22.24 | — | |
| GEMINI + DDCoTPrompting strategy=DDCoT2024.01 | 45 | 25 | 23.75 | — | |
| METER + EQSIMSetting=Fine-tuning with EQSIM regularization2023.03 | 44.99 | 22.75 | 18.75 | — | |
| METERSetting=Standard fine-tuning (FT) on Flickr30K2023.03 | 43.5 | 20.75 | 14.75 | — | |
| syn-BLIPfinetuned_on=SyViC2023.03 | 43.25 | 19.75 | 16.75 | — | |
| OPENFLAMINGO + CCoTPrompting strategy=CCoT2024.01 | 42.5 | 27.5 | 20 | — | |
| GEMINI + CoCoTPrompting strategy=CoCoT2024.01 | 40 | 32.5 | 27.75 | — | |
| METERSetting=Direct evaluation after pre-training2023.03 | 39.25 | 15.75 | 11.99 | — | |
| OPENFLAMINGOPrompting strategy=Standard2024.01 | 39 | 41.25 | 33.25 | — | |
| BLIP2023.03 | 38 | 18.25 | 14.5 | — | |
| GEMINIPrompting strategy=Standard2024.01 | 30.75 | 26 | 25 | — | |
| syn-CyCLIPfinetuned_on=SyViC2023.03 | 30 | 10.75 | 8.25 | — | |
| CyCLIP2023.03 | 28.5 | 9.5 | 7.25 | — | |
| Random Choice2024.01 | 25 | 25 | 16.67 | — | |
| GEMINI + CCoTPrompting strategy=CCoT2024.01 | 22.5 | 33 | 20.75 | — | |
| Aesthetics2026.01 | — | — | — | 24.1 | |
| GenEval2026.01 | — | — | — | 24.6 | |
| LYP2026.01 | — | — | — | 22.9 | |
| MA-PAPSPMethod Setup=MA-PAPSP, Backbone Model=SigLIP SO-400M2026.01 | — | — | — | 19.6 | |
| MA-PAPSPsize=S2026.01 | — | — | — | 22.8 | |
| MA-PAPSPsize=L2026.01 | — | — | — | 19.6 | |
| MA-PAPSP-LModel Scale=Large, Memory Augmentation=true2026.01 | — | — | — | 19.6 | |
| MA-PAPSP-SModel Scale=Small, Memory Augmentation=true2026.01 | — | — | — | 22.8 | |
| PAPSPsize=S2026.01 | — | — | — | 25.8 | |
| PAPSPsize=L2026.01 | — | — | — | 20.8 | |
| PAPSP-LModel Scale=Large, Memory Augmentation=false2026.01 | — | — | — | 20.8 | |
| PAPSP-SModel Scale=Small, Memory Augmentation=false2026.01 | — | — | — | 25.8 | |
| Qwen-VL-72BMethod Setup=LVLM-PAPSP, Backbone Model=Qwen-VL-72B2026.01 | — | — | — | 20.3 | |
| Qwen-VL-7BMethod Setup=LVLM-PAPSP, Backbone Model=Qwen-VL-7B2026.01 | — | — | — | 20.9 | |
| ReCoVERR2026.01 | — | — | — | 25.2 | |
| SeeTRUE2026.01 | — | — | — | 24.2 | |
| VQAScore2026.01 | — | — | — | 24 |