Compositional Image Retrieval on GeneCIS (test)
22.7Focus Attribute R@1OSrCIR
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| OSrCIRBackbone=ViT-G/14, Training-free=true, MLLM=GPT-4o2024.12 | 22.7 | 36.4 | 47 | 17.9 | 30.8 | 42 | 16.9 | 28.4 | 36.7 | 21 | 33.4 | 44.2 | 19.6 | |
| OSrCIRBackbone=OpenCLIP ViT-G/14, Model Configuration=w/ GPT4o2026.01 | 22.7 | 36.4 | 47 | 17.9 | 30.8 | 42 | 16.9 | 28.4 | 36.7 | 21 | 33.4 | 44.2 | — | |
| ParacosmBackbone=OpenCLIP ViT-G/142026.01 | 21.4 | 34.5 | 45.8 | 16.9 | 26.8 | 36.3 | 18.2 | 29.3 | 37.5 | 20.8 | 32.2 | 41.5 | — | |
| OSrCIRBackbone=ViT-L/14, Training-free=true, MLLM=GPT-4o2024.12 | 20.9 | 33.1 | 44.5 | 17.2 | 28.5 | 37.9 | 15 | 23.6 | 34.2 | 18.4 | 30.6 | 38.3 | 17.9 | |
| CIReVL*Backbone=ViT-G/14, Training-free=true, MLLM=GPT-4o (two-stage)2024.12 | 20.9 | 34.4 | 44.9 | 16.5 | 29 | 39.8 | 15.1 | 25.6 | 33.4 | 18.5 | 31.6 | 41.4 | 17.8 | |
| CIReVLBackbone=ViT-G/14, Training-free=true2024.12 | 20.5 | 34 | 44.5 | 16.1 | 28.6 | 39.4 | 14.7 | 25.2 | 33 | 18.1 | 31.2 | 41 | 17.4 | |
| CIReVLBackbone=OpenCLIP ViT-G/142026.01 | 20.5 | 34 | 44.5 | 16.1 | 28.6 | 39.4 | 14.7 | 25.2 | 33 | 18.1 | 31.2 | 41 | — | |
| CIReVLBackbone=ViT-G/14*2023.10 | 20.5 | 34 | 44.5 | 16.1 | 28.6 | 39.4 | 14.7 | 25.2 | 33 | 18.1 | 31.2 | 41 | 17.4 | |
| CIReVL*Backbone=ViT-L/14, Training-free=true, MLLM=GPT-4o (two-stage)2024.12 | 19.7 | 32.1 | 42.3 | 14.8 | 26.2 | 35.4 | 12.5 | 22.1 | 30.7 | 17.3 | 29.1 | 37.9 | 16.1 | |
| CIReVLBackbone=ViT-L/14, Training-free=true2024.12 | 19.5 | 31.8 | 42 | 14.4 | 26 | 35.2 | 12.3 | 21.8 | 30.5 | 17.2 | 28.9 | 37.6 | 15.9 | |
| CIReVLBackbone=ViT-L/142023.10 | 19.5 | 31.8 | 42 | 14.4 | 26 | 35.2 | 12.3 | 21.8 | 30.5 | 17.2 | 28.9 | 37.6 | 15.9 | |
| OSrCIRBackbone=ViT-B/32, Training-free=true, MLLM=GPT-4o2024.12 | 19.4 | 32.7 | 42.8 | 16.4 | 27.7 | 38.1 | 15.7 | 25.7 | 35.8 | 18.2 | 30.1 | 39.4 | 17.4 | |
| LinCIRBackbone=ViT-G/14, Training-free=false2024.12 | 19.1 | 33 | 42.3 | 17.6 | 30.2 | 38.1 | 10.1 | 19.1 | 28.1 | 7.9 | 16.3 | 25.7 | 13.7 | |
| OSrCIRBackbone=OpenCLIP ViT-G/14, Model Configuration=w/ GPT4o, Evaluation Protocol=reproduced2026.01 | 19.1 | 30.4 | 41.3 | 13.2 | 23.3 | 32.9 | 13.5 | 23.9 | 32.7 | 12.8 | 24.8 | 34.5 | — | |
| CIReVLBackbone=RN50x42023.10 | 19.1 | 31.3 | 41.5 | 14.7 | 26.3 | 37 | 13.2 | 23.4 | 32.7 | 16.3 | 28.7 | 37.9 | 15.8 | |
| Combiner (CC3M)Backbone=RN50x42023.10 | 19 | 31 | 41.5 | 16.6 | 27.5 | 36.5 | 14.7 | 25.9 | 36.1 | 16.8 | 29.1 | 39.7 | 16.8 | |
| SEARLEBackbone=ViT-B/32, Training-free=false2024.12 | 18.9 | 30.6 | 41.2 | 13 | 23.8 | 33.7 | 12.2 | 23 | 33.3 | 13.6 | 23.8 | 33.3 | 14.4 | |
| SEARLEBackbone=ViT-B/322023.10 | 18.9 | 30.6 | 41.2 | 13 | 23.8 | 33.7 | 12.2 | 23 | 33.3 | 13.6 | 23.8 | 33.3 | 14.4 | |
| CIReVL*Backbone=ViT-B/32, Training-free=true, MLLM=GPT-4o (two-stage)2024.12 | 18.2 | 29.7 | 40.7 | 15.1 | 26.1 | 36.1 | 14.9 | 24.5 | 33.6 | 16.4 | 28.1 | 37.9 | 16.2 | |
| CIReVLBackbone=ViT-B/32, Training-free=true2024.12 | 17.9 | 29.4 | 40.4 | 14.8 | 25.8 | 35.8 | 14.6 | 24.3 | 33.3 | 16.1 | 27.8 | 37.6 | 15.9 | |
| CIReVLBackbone=ViT-B/322023.10 | 17.9 | 29.4 | 40.4 | 14.8 | 25.8 | 35.8 | 14.6 | 24.3 | 33.3 | 16.1 | 27.8 | 37.6 | 15.9 | |
| Image OnlyBackbone=RN50x42023.10 | 17.7 | 30.9 | 41.9 | 11.9 | 20.8 | 28.8 | 9.3 | 18.2 | 26.2 | 7.2 | 16.7 | 24.9 | 11.5 | |
| Context-I2WBackbone=ViT-L/14, Training-free=false2024.12 | 17.2 | 30.5 | 41.7 | 16.4 | 28.3 | 37.1 | 8.7 | 17.9 | 26.9 | 7.7 | 16 | 25.4 | 12.7 | |
| SEARLEBackbone=ViT-L/14, Training-free=false2024.12 | 17.1 | 29.6 | 40.7 | 16.3 | 25.2 | 34.2 | 12 | 22.2 | 30.9 | 12 | 24.1 | 33.9 | 14.4 | |
| SEARLEBackbone=ViT-L/142023.10 | 17.1 | 29.6 | 40.7 | 16.3 | 25.2 | 34.2 | 12 | 22.2 | 30.9 | 12 | 24.1 | 33.9 | 14.4 | |
| LinCIRBackbone=ViT-L/14, Training-free=false2024.12 | 16.9 | 30 | 41.5 | 16.2 | 28 | 36.8 | 8.3 | 17.4 | 26.2 | 7.4 | 15.7 | 25 | 12.2 | |
| SEARLEBackbone=OpenCLIP ViT-G/142026.01 | 16.3 | 29.4 | 40.7 | 16.2 | 27.3 | 35.5 | 10.8 | 18.2 | 27.9 | 8.3 | 15.6 | 25.8 | — | |
| Image + TextBackbone=RN50x42023.10 | 15.6 | 26.3 | 37.1 | 12.6 | 22.9 | 32 | 10.8 | 21 | 31.2 | 11.3 | 21.5 | 30.3 | 12.6 | |
| Combiner (CIRR)Backbone=RN50x42023.10 | 15.1 | 27.7 | 39.8 | 12.1 | 22.8 | 31.8 | 13.5 | 25.4 | 36.7 | 15.4 | 28 | 39.6 | 14 | |
| Pic2WordBackbone=OpenCLIP ViT-G/142026.01 | 12.5 | 23.4 | 33.7 | 11.7 | 21.9 | 30.9 | 9.9 | 19.3 | 27.4 | 8.6 | 18.2 | 26.1 | — | |
| Text OnlyBackbone=RN50x42023.10 | 10.2 | 20.5 | 29.6 | 9.5 | 17.6 | 26.4 | 6.5 | 16.8 | 22.4 | 6.2 | 13.9 | 21.4 | 8.1 |