Image Classification on CLEVR
76.2AccuracyCLIP
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CLIPPre-training Data=WIT-400M, Backbone=ViT-L/14, Evaluation Protocol=Linear Probe2024.07 | 76.2 | — | |
| CLIP+Pre-training Data=WIT-400M, Backbone=ViT-L/14, Evaluation Protocol=Linear Probe2024.07 | 61.8 | — | |
| MLCDPre-training Data=LAION-400M, Backbone=ViT-L/14, Evaluation Protocol=Linear Probe2024.07 | 57.9 | — | |
| OpenCLIPPre-training Data=LAION-400M, Backbone=ViT-L/14, Evaluation Protocol=Linear Probe2024.07 | 57.6 | — | |
| UNICOMPre-training Data=LAION-400M, Backbone=ViT-L/14, Evaluation Protocol=Linear Probe2024.07 | 57.4 | — | |
| VP (white-box)Access=white-box, Shots=16, Repeated runs=32023.03 | 40.8 | — | |
| BlackVIPShots=16, Repeated runs=32023.03 | 36.8 | — | |
| VP w/ SPSA-GCShots=16, Repeated runs=3, Optimizer=SPSA-GC2023.03 | 25.8 | — | |
| CLIP (reported)Pre-training Data=WIT-400M, Backbone=ViT-L/14, Evaluation Protocol=Zero-shot classification2024.07 | 24.3 | — | |
| FLIPPre-training Data=LAION-400M, Backbone=ViT-L/14, Evaluation Protocol=Zero-shot classification2024.07 | 22.7 | — | |
| CLIPPre-training Data=WIT-400M, Backbone=ViT-L/14, Evaluation Protocol=Zero-shot classification2024.07 | 22.2 | — | |
| MLCDPre-training Data=LAION-400M, Backbone=ViT-L/14, Evaluation Protocol=Zero-shot classification2024.07 | 22.2 | — | |
| OpenCLIPPre-training Data=LAION-400M, Backbone=ViT-L/14, Evaluation Protocol=Zero-shot classification2024.07 | 22 | — | |
| BARShots=16, Repeated runs=32023.03 | 18.7 | — | |
| xCLIPBackbone=ViT-B/16, Pre-trained on=IT35M, Mode=Zero-shot2022.10 | 18.4 | — | |
| ZSMode=Zero-shot2023.03 | 14.5 | — | |
| Codebook-CLIPPre-trained=DataComp-10M, Evaluation Protocol=zero-shot2024.04 | 14 | — | |
| IL-CLIPPre-trained=DataComp-10M, Evaluation Protocol=zero-shot2024.04 | 13 | — | |
| CLIPBackbone=ViT-B/16, Pre-trained on=IT35M, Mode=Zero-shot2022.10 | 12.8 | — | |
| nCLIPBackbone=ViT-B/16, Pre-trained on=IT35M, Mode=Zero-shot2022.10 | 12.4 | — | |
| CLIPPre-trained=DataComp-10M, Evaluation Protocol=zero-shot2024.04 | 12 | — | |
| NegCLIPPre-trained=DataComp-10M, Evaluation Protocol=zero-shot2024.04 | 12 | — | |
| BARShots=16, Model Access Type=Black-box2024.07 | — | 18.7 | |
| BlackVIPShots=16, Model Access Type=Black-box2024.07 | — | 36.8 | |
| BlackVIP-SEShots=16, Model Access Type=Black-box2024.07 | — | 32.1 | |
| CLIPLinear probing=true, Backbone=ViT-B/16, Pre-train=IT35M2022.10 | — | 50.3 | |
| CLIPPre-training Dataset=CC3M, Backbone=ViT-B/32, Zero-shot=true2024.04 | — | 19.5 | |
| CLIPPre-training Dataset=CC12M, Backbone=ViT-B/32, Zero-shot=true2024.04 | — | 20 | |
| Codebook-CLIPPre-training Dataset=CC3M, Backbone=ViT-B/32, Zero-shot=true2024.04 | — | 19.8 | |
| Codebook-CLIPPre-training Dataset=CC12M, Backbone=ViT-B/32, Zero-shot=true2024.04 | — | 24 | |
| IL-CLIPPre-training Dataset=CC3M, Backbone=ViT-B/32, Zero-shot=true2024.04 | — | 15.8 | |
| IL-CLIPPre-training Dataset=CC12M, Backbone=ViT-B/32, Zero-shot=true2024.04 | — | 20.6 | |
| nCLIPLinear probing=true, Backbone=ViT-B/16, Pre-train=IT35M2022.10 | — | 44.6 | |
| NegCLIPPre-training Dataset=CC3M, Backbone=ViT-B/32, Zero-shot=true2024.04 | — | 15.1 | |
| NegCLIPPre-training Dataset=CC12M, Backbone=ViT-B/32, Zero-shot=true2024.04 | — | 17.5 | |
| VP (white-box)Shots=16, Model Access Type=White-box2024.07 | — | 40.8 | |
| VP w/ SPSA-GCShots=16, Model Access Type=Black-box2024.07 | — | 25.8 | |
| xCLIPLinear probing=true, Backbone=ViT-B/16, Pre-train=IT35M2022.10 | — | 49.8 | |
| ZSEvaluation Protocol=Zero-shot2024.07 | — | 14.5 |