# Image-to-text retrieval on SugarCrepe (Compositional R@1) Source: [Image-to-text retrieval on SugarCrepe (Compositional R@1)]() Collection index: [llms.txt]() Data updated: 2026-05-26T10:06:39.991Z Evaluation of Image-to-text retrieval performance on the SugarCrepe dataset, specifically measuring retrieval success using Add, Replace, and Swap compositional metrics (R@1). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Image-to-text retrieval | | Dataset (including any stated split) | SugarCrepe | | Metric | R@1 (Add) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 37 of 37 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [R@1 (Add)](): Higher is better; unit: PERCENT. - [R@1 (Replace)](): Higher is better; unit: PERCENT. - [R@1 (Swap)](): Higher is better; unit: PERCENT. - [R@1 (Replace, obj)](): Higher is better; unit: PERCENT. - [R@1 (Replace, att)](): Higher is better; unit: PERCENT. - [R@1 (Replace, rel)](): Higher is better; unit: PERCENT. - [R@1 (Swap, obj)](): Higher is better; unit: PERCENT. - [R@1 (Swap, att)](): Higher is better; unit: PERCENT. - [R@1 (Add, obj)](): Higher is better; unit: PERCENT. - [R@1 (Add, att)](): Higher is better; unit: PERCENT. - [R@1 (Replace, Avg)](): Higher is better; unit: PERCENT. - [R@1 (Swap, Avg)](): Higher is better; unit: PERCENT. - [R@1 (Add, Avg)](): Higher is better; unit: PERCENT. ## Recorded results | Method | R@1 (Add) | Result date | Report paper | | --- | --- | --- | --- | | JAM | 98.44 | Not recorded | 68fa7afd1e209131cea861a5 | | JAM | 97.83 | Not recorded | 68fa7afd1e209131cea861a5 | | JAM | 97.41 | Not recorded | 68fa7afd1e209131cea861a5 | | JAM | 96.85 | Not recorded | 68fa7afd1e209131cea861a5 | | JAM | 96.85 | Not recorded | 68fa7afd1e209131cea861a5 | | JAM | 96.67 | Not recorded | 68fa7afd1e209131cea861a5 | | JAM | 95.74 | Not recorded | 68fa7afd1e209131cea861a5 | | JAM | 94.81 | Not recorded | 68fa7afd1e209131cea861a5 | | JAM | 94.8 | Not recorded | 68fa7afd1e209131cea861a5 | | JAM | 94.25 | Not recorded | 68fa7afd1e209131cea861a5 | | JAM | 94.1 | Not recorded | 68fa7afd1e209131cea861a5 | | Finetuned CLIP (ViT-B/32) | 92.69 | Not recorded | 68fa7afd1e209131cea861a5 | | JAM | 90.12 | Not recorded | 68fa7afd1e209131cea861a5 | | Pretrained CLIP (ViT-B-32/SigLIP) | 86.56 | Not recorded | 68fa7afd1e209131cea861a5 | | Pretrained CLIP (ViT-B-32/LAION-400m) | 79.64 | Not recorded | 68fa7afd1e209131cea861a5 | | Pretrained CLIP (ViT-B-32/OpenAI) | 77.58 | Not recorded | 68fa7afd1e209131cea861a5 | | IL-CLIP | 73.8 | Not recorded | 68fa72f41e209131cea598b9 | | Codebook-CLIP | 71.3 | Not recorded | 68fa72f41e209131cea598b9 | | JAM | 67.94 | Not recorded | 68fa7afd1e209131cea861a5 | | CLIP | 67.5 | Not recorded | 68fa72f41e209131cea598b9 | | IL-CLIP | 66.1 | Not recorded | 68fa72f41e209131cea598b9 | | Codebook-CLIP | 65.9 | Not recorded | 68fa72f41e209131cea598b9 | | JAM | 65.17 | Not recorded | 68fa7afd1e209131cea861a5 | | NegCLIP | 65 | Not recorded | 68fa72f41e209131cea598b9 | | JAM | 64.35 | Not recorded | 68fa7afd1e209131cea861a5 | | JAM | 63.02 | Not recorded | 68fa7afd1e209131cea861a5 | | CLIP | 61.9 | Not recorded | 68fa72f41e209131cea598b9 | | JAM | 60.99 | Not recorded | 68fa7afd1e209131cea861a5 | | NegCLIP | 59.3 | Not recorded | 68fa72f41e209131cea598b9 | | JAM | 58.34 | Not recorded | 68fa7afd1e209131cea861a5 | ### JAM R@1 (Add): 98.44. Result ID: 6a0ab68b47f34ce924f04354. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language Backbone":"OLMo2 (7B)","Vision Backbone":"ResNet50","Alignment Method":"Spread"}. This record does not assert independent reproduction. ### JAM R@1 (Add): 97.83. Result ID: 6a0ab68b47f34ce924f0434b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language Backbone":"Llama3.2 (1B)","Vision Backbone":"DINOv2 (ViT-B)","Alignment Method":"Spread"}. This record does not assert independent reproduction. ### JAM R@1 (Add): 97.41. Result ID: 6a0ab68b47f34ce924f04351. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language Backbone":"OLMo2 (7B)","Vision Backbone":"DINOv2 (ViT-B)","Alignment Method":"Spread"}. This record does not assert independent reproduction. ### JAM R@1 (Add): 96.85. Result ID: 6a0ab68b47f34ce924f04344. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language Backbone":"Gemma2 (2B)","Vision Backbone":"DINOv2 (ViT-B; 86M)","Alignment Method":"NegCon"}. This record does not assert independent reproduction. ### JAM R@1 (Add): 96.85. Result ID: 6a0ab68b47f34ce924f0434e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language Backbone":"Llama3.2 (1B)","Vision Backbone":"ResNet50","Alignment Method":"Spread"}. This record does not assert independent reproduction. ### JAM R@1 (Add): 96.67. Result ID: 6a0ab68b47f34ce924f04350. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language Backbone":"OLMo2 (7B)","Vision Backbone":"DINOv2 (ViT-B)","Alignment Method":"NegCon"}. This record does not assert independent reproduction. ### JAM R@1 (Add): 95.74. Result ID: 6a0ab68b47f34ce924f04345. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language Backbone":"Gemma2 (2B)","Vision Backbone":"DINOv2 (ViT-B; 86M)","Alignment Method":"Spread"}. This record does not assert independent reproduction. ### JAM R@1 (Add): 94.81. Result ID: 6a0ab68b47f34ce924f04348. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language Backbone":"Gemma2 (2B)","Vision Backbone":"ResNet50","Alignment Method":"Spread"}. This record does not assert independent reproduction. ### JAM R@1 (Add): 94.8. Result ID: 6a0ab68b47f34ce924f04347. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language Backbone":"Gemma2 (2B)","Vision Backbone":"ResNet50","Alignment Method":"NegCon"}. This record does not assert independent reproduction. ### JAM R@1 (Add): 94.25. Result ID: 6a0ab68b47f34ce924f0434a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language Backbone":"Llama3.2 (1B)","Vision Backbone":"DINOv2 (ViT-B)","Alignment Method":"NegCon"}. This record does not assert independent reproduction. ### JAM R@1 (Add): 94.1. Result ID: 6a0ab68b47f34ce924f0434d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language Backbone":"Llama3.2 (1B)","Vision Backbone":"ResNet50","Alignment Method":"NegCon"}. This record does not assert independent reproduction. ### Finetuned CLIP (ViT-B/32) R@1 (Add): 92.69. Result ID: 6a0ab68b47f34ce924f04358. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"ViT-B/32","mode":"fine-tuned"}. This record does not assert independent reproduction. ### JAM R@1 (Add): 90.12. Result ID: 6a0ab68b47f34ce924f04353. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language Backbone":"OLMo2 (7B)","Vision Backbone":"ResNet50","Alignment Method":"NegCon"}. This record does not assert independent reproduction. ### Pretrained CLIP (ViT-B-32/SigLIP) R@1 (Add): 86.56. Result ID: 6a0ab68b47f34ce924f04357. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"ViT-B-32","Training Data":"SigLIP"}. This record does not assert independent reproduction. ### Pretrained CLIP (ViT-B-32/LAION-400m) R@1 (Add): 79.64. Result ID: 6a0ab68b47f34ce924f04356. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"ViT-B-32","Training Data":"LAION-400m"}. This record does not assert independent reproduction. ### Pretrained CLIP (ViT-B-32/OpenAI) R@1 (Add): 77.58. Result ID: 6a0ab68b47f34ce924f04355. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"ViT-B-32","Training Data":"OpenAI"}. This record does not assert independent reproduction. ### IL-CLIP R@1 (Add): 73.8. Result ID: 6992a50f3a6c314849105da4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Pre-training Dataset":"CC12M"}. This record does not assert independent reproduction. ### Codebook-CLIP R@1 (Add): 71.3. Result ID: 6992a50f3a6c314849105da2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Pre-training Dataset":"CC12M"}. This record does not assert independent reproduction. ### JAM R@1 (Add): 67.94. Result ID: 6a0ab68b47f34ce924f0434c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language Backbone":"Llama3.2 (1B)","Vision Backbone":"ResNet50","Alignment Method":"Con"}. This record does not assert independent reproduction. ### CLIP R@1 (Add): 67.5. Result ID: 6992a50f3a6c314849105da1. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Pre-training Dataset":"CC12M"}. This record does not assert independent reproduction. ### IL-CLIP R@1 (Add): 66.1. Result ID: 6992a50f3a6c314849105da0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Pre-training Dataset":"CC3M"}. This record does not assert independent reproduction. ### Codebook-CLIP R@1 (Add): 65.9. Result ID: 6992a50f3a6c314849105d9e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Pre-training Dataset":"CC3M"}. This record does not assert independent reproduction. ### JAM R@1 (Add): 65.17. Result ID: 6a0ab68b47f34ce924f0434f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language Backbone":"OLMo2 (7B)","Vision Backbone":"DINOv2 (ViT-B)","Alignment Method":"Con"}. This record does not assert independent reproduction. ### NegCLIP R@1 (Add): 65. Result ID: 6992a50f3a6c314849105da3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Pre-training Dataset":"CC12M"}. This record does not assert independent reproduction. ### JAM R@1 (Add): 64.35. Result ID: 6a0ab68b47f34ce924f04352. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language Backbone":"OLMo2 (7B)","Vision Backbone":"ResNet50","Alignment Method":"Con"}. This record does not assert independent reproduction. ### JAM R@1 (Add): 63.02. Result ID: 6a0ab68b47f34ce924f04346. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language Backbone":"Gemma2 (2B)","Vision Backbone":"ResNet50","Alignment Method":"Con"}. This record does not assert independent reproduction. ### CLIP R@1 (Add): 61.9. Result ID: 6992a50f3a6c314849105d9d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Pre-training Dataset":"CC3M"}. This record does not assert independent reproduction. ### JAM R@1 (Add): 60.99. Result ID: 6a0ab68b47f34ce924f04349. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language Backbone":"Llama3.2 (1B)","Vision Backbone":"DINOv2 (ViT-B)","Alignment Method":"Con"}. This record does not assert independent reproduction. ### NegCLIP R@1 (Add): 59.3. Result ID: 6992a50f3a6c314849105d9f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Pre-training Dataset":"CC3M"}. This record does not assert independent reproduction. ### JAM R@1 (Add): 58.34. Result ID: 6a0ab68b47f34ce924f04343. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language Backbone":"Gemma2 (2B)","Vision Backbone":"DINOv2 (ViT-B; 86M)","Alignment Method":"Con"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.