Compositional Reasoning on VL-Checklist
81.8Attribute ScoreSGVL
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| SGVLBackbone=BLIP2023.05 | 81.8 | 85.2 | 81.9 | — | |
| SGVLBackbone=BLIP22023.05 | 81.2 | 88.4 | 88.8 | — | |
| BLIP2Architecture=Encoder-decoder2023.05 | 80.12 | 84.14 | 70.72 | 53.27 | |
| NegBLIP22023.05 | 79 | 87 | 88.2 | — | |
| NegBLIP2023.05 | 78.2 | 83.7 | 81.9 | — | |
| BLIP22023.05 | 77.8 | 84.9 | 84.9 | — | |
| DAC-LLMArchitecture=Dual-encoder, Caption Density Source=LLM-based2023.05 | 77.27 | 87.3 | 86.41 | 85.18 | |
| DAC-SAMArchitecture=Dual-encoder, Caption Density Source=SAM-based2023.05 | 75.829 | 88.5 | 89.75 | 83.83 | |
| syn-BLIPfinetuned_on=SyViC2023.03 | 75.34 | — | 70.18 | 72.76 | |
| BLIP2023.05 | 75.2 | 82.2 | 81.5 | — | |
| MosaiCLIP NoCurriccurriculum_learning=false2023.05 | 75 | 86.4 | 69.6 | — | |
| MosaiCLIPFine-tuning data=CC3M2023.05 | 73.7 | 86.4 | 71.9 | — | |
| MosaiCLIP2023.05 | 73.7 | 86.4 | 71.9 | — | |
| MosaiCLIP WiSE-FTfine_tuning=WiSE-FT2023.05 | 73.6 | 86.5 | 72.2 | — | |
| ViLT2023.05 | 73.3 | 85 | 62 | — | |
| BLIPtraining_data=LAION2023.03 | 73.11 | — | 68.45 | 70.78 | |
| NegClipArchitecture=Dual-encoder2023.05 | 72.236 | 81.35 | 63.525 | 78.01 | |
| MACCO-CLIPBackbone=CLIP ViT-L/142026.06 | 72.1 | — | 72.5 | — | |
| Teaching SVLCFine-tuning data=CC3M2023.05 | 72 | 85 | 69 | — | |
| Teaching SVLCaugmentation=External LLM text augmentation2023.05 | 72 | 85 | 69 | — | |
| SGVLBackbone=CLIP2023.05 | 72 | 82.6 | 82 | — | |
| SVLCArchitecture=Dual-encoder2023.05 | 71.97 | 85 | 68.95 | 79.42 | |
| CLIP-FTBackbone=CLIP ViT-L/142026.06 | 71.8 | — | 63 | — | |
| MiniGPT-42023.05 | 71.3 | 84.2 | 84.1 | — | |
| MACCO-SigLIPBackbone=SigLIP ViT-B/162026.06 | 71.2 | — | 63.7 | — | |
| Syn-CLIPFine-tuning data=CC3M2023.05 | 70.4 | — | 69.4 | — | |
| Syn-CLIPtraining_data=Million-scale synthetic data2023.05 | 70.4 | — | 69.4 | — | |
| syn-CLIPtraining=finetuned on SyViC2023.03 | 70.37 | — | 69.39 | 69.88 | |
| SigLIP-FTBackbone=SigLIP ViT-B/162026.06 | 69.9 | — | 60.6 | — | |
| MACCO-CLIPBackbone=CLIP ViT-B/162026.06 | 68.9 | — | 66.5 | — | |
| CLIPBackbone=CLIP ViT-B/162026.06 | 68.8 | — | 61.7 | — | |
| CLIP-FTBackbone=CLIP ViT-B/162026.06 | 68.5 | — | 56.9 | — | |
| syn-CyCLIPtraining=finetuned on SyViC2023.03 | 68.06 | — | 65.73 | 66.89 | |
| NegCLIP2023.05 | 68 | 81.4 | 81.3 | — | |
| CLIPBackbone=CLIP ViT-L/142026.06 | 68 | — | 64.7 | — | |
| CLIPArchitecture=Dual-encoder2023.05 | 67.6 | 81.38 | 63.05 | 63.35 | |
| CLIPFine-tuning data=CC3M2023.05 | 67.6 | 81.6 | 63.1 | — | |
| CLIPmode=Zero-shot2023.05 | 67.6 | 81.6 | 63.1 | — | |
| CLIPtraining=pre-trained on large-scale real data2023.03 | 67.51 | — | 63.57 | 65.54 | |
| CyCLIPtraining=pre-trained on large-scale real data2023.03 | 66.96 | — | 61.15 | 64.06 | |
| LLaVA2023.05 | 65.5 | 83.1 | 83 | — | |
| CLIP2023.05 | 65.5 | 80.6 | 78 | — | |
| CLIP-FTFine-tuning data=CC3M2023.05 | 64.7 | 79 | 54.3 | — | |
| CLIP-FTmode=Fine-tuned2023.05 | 64.7 | 79 | 54.3 | — | |
| SigLIPBackbone=SigLIP ViT-B/162026.06 | 59.8 | — | 42 | — | |
| FLAVA2023.05 | 54.6 | 70.6 | 46.6 | — | |
| Random Chance2026.06 | 50 | — | 50 | — | |
| [18]Training Source=CC3M, Num Synthetic Captions=9M2025.03 | — | — | — | 75.3 | |
| [79]Training Source=COCO, Num Synthetic Images=82K, Num Synthetic Captions=82K2025.03 | — | — | — | 69.9 | |
| CE-CLIPEvaluation Protocol=Zero-shot2023.06 | — | — | — | 75.1 | |
| CE-CLIPTraining Source=COCO, Num Synthetic Captions=2M2025.03 | — | — | — | 76.3 | |
| CE-CLIP+Evaluation Protocol=Zero-shot2023.06 | — | — | — | 78.4 | |
| CE-CLIP+Training Source=COCO+CC3M, Num Synthetic Captions=15M2025.03 | — | — | — | 79.3 | |
| CLIPEvaluation Protocol=Zero-shot2023.06 | — | — | — | 69.2 | |
| CLIPTraining Protocol=Zero-Shot2025.03 | — | — | — | 73.2 | |
| CLIPTraining Source=COCO, Training Protocol=Finetune2025.03 | — | — | — | 72.8 | |
| CLIP-FTEvaluation Protocol=Fine-tuned on COCO2023.06 | — | — | — | 68.6 | |
| COMOTraining Source=COCO, Num Synthetic Images=567K, Num Synthetic Captions=567K2025.03 | — | — | — | 76.9 | |
| FSC-CLIPTraining Source=COCO, Num Synthetic Captions=1.5M2025.03 | — | — | — | 77.2 | |
| MosaiCLIPTraining Source=COCO, Num Synthetic Captions=981K2025.03 | — | — | — | 76.8 | |
| NegCLIPTraining Source=COCO, Num Synthetic Captions=500K2025.03 | — | — | — | 74.6 | |
| SPARCLTraining Source=COCO, Num Synthetic Images=820K, Num Synthetic Captions=820K2025.03 | — | — | — | 79.2 | |
| syn-CLIPTraining Source=SyViC, Num Synthetic Images=>1M, Num Synthetic Captions=>1M2025.03 | — | — | — | 74.8 |