Domain Conversion Retrieval on ImageNet-R
27.54Recall@10FREEDOM
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| FREEDOM# PARAM=428M, Training status=Training-free2024.12 | 27.54 | — | — | — | — | — | — | — | — | — | — | 47.05 | |
| CIReVL# PARAM=12.5B, Training status=Training-free2024.12 | 23.75 | — | — | — | — | — | — | — | — | — | — | 43.05 | |
| KEDs# PARAM=428M, Training status=Requires training2024.12 | 18 | — | — | — | — | — | — | — | — | — | — | 35.4 | |
| SBCIR2024.03 | 15.3 | — | — | — | — | — | — | — | — | — | — | 27.1 | |
| CompoDiff (CLIP+T5-XL)# PARAM=3.6B, Training status=Requires training2024.12 | 13.31 | — | — | — | — | — | — | — | — | — | — | 24.86 | |
| CompoDiff (CLIP)# PARAM=568M, Training status=Requires training2024.12 | 13.18 | — | — | — | — | — | — | — | — | — | — | 24.86 | |
| Context-I2W# PARAM=496M, Training status=Requires training2024.12 | 12.9 | — | — | — | — | — | — | — | — | — | — | 27.6 | |
| ARTEMIS w/ CompoDiff dataset# PARAM=33M, Training status=Requires training2024.12 | 12.24 | — | — | — | — | — | — | — | — | — | — | 23.58 | |
| WeiCom# PARAM=428M, Training status=Training-free2024.12 | 12.17 | — | — | — | — | — | — | — | — | — | — | 23.04 | |
| CLIP4Cir w/ CompoDiff dataset# PARAM=178M, Training status=Requires training2024.12 | 12.11 | — | — | — | — | — | — | — | — | — | — | 23.79 | |
| SEARLE2024.03 | 12.1 | — | — | — | — | — | — | — | — | — | — | 25.4 | |
| SEARLE# PARAM=428M, Training status=Training-free2024.12 | 11.76 | — | — | — | — | — | — | — | — | — | — | 30.96 | |
| MagicLens# PARAM=465M, Training status=Training-free2024.12 | 11.34 | — | — | — | — | — | — | — | — | — | — | 26.78 | |
| CompoDiff (T5-XL)# PARAM=3B, Training status=Requires training2024.12 | 11.14 | — | — | — | — | — | — | — | — | — | — | 21.76 | |
| Pic2Word2024.03 | 10.8 | — | — | — | — | — | — | — | — | — | — | 22.7 | |
| Pic2Word (LAION 2B-en)# PARAM=429M, Training status=Requires training2024.12 | 10.35 | — | — | — | — | — | — | — | — | — | — | 22.91 | |
| Pic2Word (CC-3M)# PARAM=429M, Training status=Requires training2024.12 | 10.19 | — | — | — | — | — | — | — | — | — | — | 22.7 | |
| Pic2Word# PARAM=429M, Training status=Requires training2024.12 | 10.05 | — | — | — | — | — | — | — | — | — | — | 23.23 | |
| SceneTrilogy2024.03 | 9.7 | — | — | — | — | — | — | — | — | — | — | 20.5 | |
| MagicLens (original prompt)# PARAM=465M, Training status=Training-free2024.12 | 8.61 | — | — | — | — | — | — | — | — | — | — | 21.6 | |
| Combiner2024.03 | 8.3 | — | — | — | — | — | — | — | — | — | — | 19.8 | |
| TASK-former2024.03 | 7.6 | — | — | — | — | — | — | — | — | — | — | 18.2 | |
| SEARLE (default)# PARAM=428M, Training status=Training-free2024.12 | 2.36 | — | — | — | — | — | — | — | — | — | — | 14.24 | |
| B-CLIPinput_mode=Sketch + Text2024.03 | 2.2 | — | — | — | — | — | — | — | — | — | — | 12.9 | |
| CombinerSupervision=CIRR2024.03 | — | 6.1 | 14.8 | 10.5 | 21.3 | 7 | 17.7 | 8.5 | 20.4 | 8 | 18.5 | — | |
| CombinerSupervision=Fashion-IQ2024.03 | — | 6 | 16.9 | 7.6 | 20.2 | 2.7 | 10.9 | 8 | 21.6 | 6 | 17.4 | — | |
| Image-onlySupervision=ZERO-SHOT2024.03 | — | 0.3 | 4.5 | 0.2 | 1.8 | 0.6 | 5.7 | 0.3 | 4 | 0.4 | 4 | — | |
| Image+TextSupervision=ZERO-SHOT2024.03 | — | 2.2 | 13.3 | 2 | 10.3 | 1.2 | 9.7 | 1.6 | 11.6 | 1.7 | 11.2 | — | |
| KEDsSupervision=ZERO-SHOT2024.03 | — | 14.8 | 34.2 | 23.5 | 34.8 | 16.5 | 36.3 | 17.4 | 36.4 | 18 | 35.4 | — | |
| Pic2wordSupervision=ZERO-SHOT2024.03 | — | 8 | 21.9 | 13.5 | 25.6 | 8.7 | 21.6 | 10 | 23.8 | 10.1 | 23.2 | — | |
| Text-onlySupervision=ZERO-SHOT2024.03 | — | 0.2 | 1.1 | 0.8 | 3.7 | 0.8 | 2.4 | 0.4 | 2 | 0.5 | 2.3 | — |