Composed Image Retrieval on CIRCO
56.54mAP@5OSCAR
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| OSCARType=Agentic, Training Free=true2026.02 | 56.54 | — | 58.53 | 62.67 | 61.92 | |
| V-Retrver-7BZero-shot=true, Query format=(q_i, q_t) -> c_i2026.02 | 48.2 | — | — | — | — | |
| U-MARVEL+Reranker=true2025.07 | 46 | — | — | — | — | |
| QQMM-embed-v2Type=Multimodal Embedding, Training Free=true2026.02 | 45.92 | — | 47.13 | 51.45 | 50.39 | |
| LamRAreranking=pointwise/listwise2024.12 | 42.8 | — | — | — | — | |
| LamRA-7BZero-shot=true, Query format=(q_i, q_t) -> c_i2026.02 | 42.8 | — | — | — | — | |
| LamRAReranker=true2025.07 | 42.8 | — | — | — | — | |
| ParacosmBackbone=OpenCLIP ViT-G/142026.01 | 39.82 | — | — | — | — | |
| ORACLE-QPolicy=ORACLE-Q, Turn budget (T)=T32026.05 | 38.1 | — | — | — | — | |
| U-MARVELReranker=false2025.07 | 36.2 | — | — | — | — | |
| Ours (EIG)Policy=Ours (EIG), Turn budget (T)=T32026.05 | 36 | — | — | — | — | |
| ORACLE-QPolicy=ORACLE-Q, Turn budget (T)=T22026.05 | 35.6 | — | — | — | — | |
| MM-Embed-7BZero-shot=true, Query format=(q_i, q_t) -> c_i2026.02 | 35.5 | — | — | — | — | |
| MagicLens-LBackbone=CoCa-L, # Total Params=613M2024.03 | 34.1 | 70.9 | — | — | — | |
| Ours (EIG)Policy=Ours (EIG), Turn budget (T)=T22026.05 | 33.4 | — | — | — | — | |
| LamRA-Ret2024.12 | 33.2 | — | — | — | — | |
| LamRA-RetReranker=false2025.07 | 33.2 | — | — | — | — | |
| RzenEmbed-v2-7BType=Multimodal Embedding, Training Free=true2026.02 | 32.2 | — | 34.19 | 38.61 | 37.41 | |
| X^RType=Agentic, Training Free=true2026.02 | 31.38 | — | 32.88 | 36.5 | 35.46 | |
| MLLM-QPolicy=MLLM-Q, Turn budget (T)=T32026.05 | 31.2 | — | — | — | — | |
| FiREType=CIR Dedicated, Training Free=false2026.02 | 31.03 | — | 32.08 | 35.5 | 34.4 | |
| ORACLE-QPolicy=ORACLE-Q, Turn budget (T)=T12026.05 | 31 | — | — | — | — | |
| MagicLens-BBackbone=CoCa-B, # Total Params=267M2024.03 | 30.8 | 69.3 | — | — | — | |
| MagicLens-LBackbone=CLIP-L, # Total Params=465M2024.03 | 29.6 | 68.1 | — | — | — | |
| MagicLens-L2024.12 | 29.6 | — | — | — | — | |
| MagicLens-LZero-shot=true, Query format=(q_i, q_t) -> c_i2026.02 | 29.6 | — | — | — | — | |
| MagicLens-LReranker=false2025.07 | 29.6 | — | — | — | — | |
| MagicLensBackbone=ViT-L/14, Training Costs=128 TPU, 6h2026.07 | 29.6 | — | 30.8 | 34.4 | 33.4 | |
| Ours (EIG)Policy=Ours (EIG), Turn budget (T)=T12026.05 | 29.1 | — | — | — | — | |
| MLLM-QPolicy=MLLM-Q, Turn budget (T)=T22026.05 | 29 | — | — | — | — | |
| FIXED-QPolicy=FIXED-Q, Turn budget (T)=T32026.05 | 28.1 | — | — | — | — | |
| MRA-CIRType=Agentic, Training Free=false2026.02 | 27.14 | — | 28.85 | 32.63 | 31.54 | |
| CIReVLBackbone=OpenCLIP ViT-G/142026.01 | 26.77 | — | — | — | — | |
| FIXED-QPolicy=FIXED-Q, Turn budget (T)=T22026.05 | 26.7 | — | — | — | — | |
| MLLM-QPolicy=MLLM-Q, Turn budget (T)=T12026.05 | 26.2 | — | — | — | — | |
| OSrCIRBackbone=OpenCLIP ViT-G/14, reproduced=true2026.01 | 25.62 | — | — | — | — | |
| RANDOM-QPolicy=RANDOM-Q, Turn budget (T)=T32026.05 | 25.6 | — | — | — | — | |
| FIXED-QPolicy=FIXED-Q, Turn budget (T)=T12026.05 | 25 | — | — | — | — | |
| RANDOM-QPolicy=RANDOM-Q, Turn budget (T)=T22026.05 | 24.9 | — | — | — | — | |
| E5-V2024.12 | 24.8 | — | — | — | — | |
| E5-VZero-shot=true, Query format=(q_i, q_t) -> c_i2026.02 | 24.8 | — | — | — | — | |
| E5-VReranker=false2025.07 | 24.8 | — | — | — | — | |
| DQE-CIRsetting=zero-shot2026.03 | 24.27 | — | 25.68 | 28.13 | 27.47 | |
| RANDOM-QPolicy=RANDOM-Q, Turn budget (T)=T12026.05 | 24.1 | — | — | — | — | |
| AutoCIRType=Agentic, Training Free=true2026.02 | 24.05 | — | 25.14 | 28.36 | 27.35 | |
| Baseline (ViT-G)Pretraining Data=OACIRR, Pretraining Scale=127 K, Zero-shot=true2026.04 | 23.96 | — | 24.69 | — | 26.58 | |
| RANDOM-QPolicy=RANDOM-Q, Turn budget (T)=T02026.05 | 23.5 | — | — | — | — | |
| FIXED-QPolicy=FIXED-Q, Turn budget (T)=T02026.05 | 23.5 | — | — | — | — | |
| MLLM-QPolicy=MLLM-Q, Turn budget (T)=T02026.05 | 23.5 | — | — | — | — | |
| Ours (EIG)Policy=Ours (EIG), Turn budget (T)=T02026.05 | 23.5 | — | — | — | — | |
| ORACLE-QPolicy=ORACLE-Q, Turn budget (T)=T02026.05 | 23.5 | — | — | — | — | |
| LDREType=CIR Dedicated, Training Free=true2026.02 | 23.35 | — | 24.03 | 27.5 | 26.44 | |
| QuResetting=zero-shot2026.03 | 23.22 | — | 24.23 | 27.24 | 26.26 | |
| MagicLens-BBackbone=CLIP-B, # Total Params=166M2024.03 | 23.1 | 66.7 | — | — | — | |
| MagicLensBackbone=ViT-B/32, Training Costs=64 TPU, 6h2026.07 | 23.1 | — | 23.8 | 26.7 | 25.8 | |
| OSrCIRSetting=Zero-shot, Turns=m=02026.06 | 22.4 | — | 23.6 | — | — | |
| CLARASetting=Zero-shot, Turns=m=02026.06 | 22.1 | — | 23.9 | — | — | |
| CoAlign (ViT-G)Pretraining Data=CIRHS, Pretraining Scale=535 K, Zero-shot=true2026.04 | 21.6 | — | 23.38 | — | 25.98 | |
| Baseline (ViT-L)Pretraining Data=OACIRR, Pretraining Scale=127 K, Zero-shot=true2026.04 | 21.51 | — | 23.14 | — | 25.47 | |
| CoVR-BLIPPretraining Data=WebVid-CoVR, Pretraining Scale=1,644 K, Zero-shot=true2026.04 | 21.43 | — | 22.33 | — | 24.47 | |
| CIReVLBackbone=CLIP-L, # Total Params=12.5B*2024.03 | 18.6 | 59.5 | — | — | — | |
| CIReVLType=CIR Dedicated, Training Free=true2026.02 | 18.57 | — | 19.01 | 21.8 | 20.89 | |
| CoVRsetting=zero-shot2026.03 | 18.35 | — | 19.25 | 21.88 | 21.02 | |
| MCL (LLaMA2-7B)Backbone=ViT-L/14, Training Costs=NA2026.07 | 17.7 | — | 18.9 | 21.7 | 20.8 | |
| SPRCsetting=zero-shot2026.03 | 17.57 | — | 18.48 | 20.98 | 20.14 | |
| Qwen3-Embed-8BType=Text Embedding, Training Free=true2026.02 | 16.45 | — | 17.23 | 19.88 | 18.98 | |
| CompoDiffPretraining Data=ST18M + LAION-2B, Pretraining Scale=18,000 K, Zero-shot=true2026.04 | 15.33 | — | 17.71 | — | 19.45 | |
| CIReVLBackbone=CLIP-B, # Total Params=12.3B*2024.03 | 14.9 | 60.2 | — | — | — | |
| FlowCIRBackbone=ViT-L/14, Training Costs=RTX 3090, 0.5h2026.07 | 14.9 | — | 15.7 | 18.2 | 17.3 | |
| Ops-MM-v1-7BType=Multimodal Embedding, Training Free=true2026.02 | 13.56 | — | 15.96 | 19.68 | 18.61 | |
| Qwen3-Embed-4BType=Text Embedding, Training Free=true2026.02 | 13.55 | — | 14.6 | 17.2 | 16.33 | |
| SEARLEBackbone=OpenCLIP ViT-G/142026.01 | 13.2 | — | — | — | — | |
| FlowCIRBackbone=ViT-B/32, Training Costs=RTX 3090, 0.5h2026.07 | 13.1 | — | 13.4 | 15.3 | 14.6 | |
| LinCIR + CIGBackbone=ViT-L/14, Training Costs=NA2026.07 | 13 | — | 13.6 | 16 | 15.1 | |
| CompoDiffBackbone=CLIP-L, # Total Params=568M2024.03 | 12.6 | 57.4 | — | — | — | |
| LinCIRBackbone=CLIP-L, # Total Params=442M2024.03 | 12.6 | 57.1 | — | — | — | |
| CompoDiffSetting=Zero-shot, Turns=m=02026.06 | 12.6 | — | 15.3 | — | — | |
| LinCIRBackbone=ViT-L/14, Training Costs=8 A100, 0.5h2026.07 | 12.6 | — | 13.6 | 15.9 | 15 | |
| Compo-DiffBackbone=ViT-L/14, Training Costs=128 A100, 231h2026.07 | 12.6 | — | 13.4 | 16.4 | 15.8 | |
| LinCIRType=CIR Dedicated, Training Free=false2026.02 | 12.59 | — | 13.58 | 15.85 | 15 | |
| UniIR-CLIP2024.12 | 12.5 | — | — | — | — | |
| UniIR-CLIPZero-shot=true, Query format=(q_i, q_t) -> c_i2026.02 | 12.5 | — | — | — | — | |
| UniIR-CLIP_SFReranker=false2025.07 | 12.5 | — | — | — | — | |
| SEARLEBackbone=CLIP-L, # Total Params=442M2024.03 | 11.7 | 53.8 | — | — | — | |
| SEARLESetting=Zero-shot, Turns=m=02026.06 | 11.7 | — | 13.1 | — | — | |
| SEARLEBackbone=ViT-L/14, Training Costs=8 A100, 4h2026.07 | 11.7 | — | 12.7 | 15.1 | 14.3 | |
| SEARLEType=CIR Dedicated, Training Free=false2026.02 | 11.68 | — | 12.73 | 15.12 | 14.33 | |
| iSEARLEBackbone=ViT-L/14, Training Costs=122026.07 | 11.3 | — | 12.7 | 15.3 | 14.5 | |
| iSEARLEBackbone=ViT-B/32, Training Costs=A100, 12h2026.07 | 10.6 | — | 11.2 | 13.3 | 12.5 | |
| CLIP4Cirsetting=zero-shot2026.03 | 10.58 | — | 11.18 | 12.96 | 12.32 | |
| PLIBackbone=CLIP-L, # Total Params=428M+2024.03 | 10.4 | 55.6 | — | — | — | |
| SEARLE + CIGBackbone=ViT-B/32, Training Costs=NA2026.07 | 10.2 | — | 10.6 | 12.5 | 11.8 | |
| SEARLE-XL-OTIType=CIR Dedicated, Training Free=false2026.02 | 10.18 | — | 11.03 | 13.67 | 12.72 | |
| Qwen3-Embed-0.6BType=Text Embedding, Training Free=true2026.02 | 9.55 | — | 10.35 | 12.33 | 11.61 | |
| Pic2WordSetting=Zero-shot, Turns=m=02026.06 | 9.5 | — | 10.6 | — | — | |
| SEARLEBackbone=CLIP-B, # Total Params=165M2024.03 | 9.4 | 54.9 | — | — | — | |
| SEARLEBackbone=ViT-B/32, Training Costs=8 A100, 4h2026.07 | 8.9 | — | 9.4 | 11.2 | 10.6 | |
| Pic2WordType=CIR Dedicated, Training Free=false2026.02 | 8.72 | — | 9.51 | 11.29 | 10.46 | |
| Pic2WordBackbone=CLIP-L, # Total Params=429M2024.03 | 8.7 | — | — | — | — | |
| Pic2WordBackbone=ViT-L/14, Training Costs=8 A100, 16h2026.07 | 8.7 | — | 9.5 | 11.3 | 10.6 |