Text Retrieval on Flickr30K
100R@1HQA-VLAttack
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| HQA-VLAttackSurrogate Model=CLIPViT, Victim Model=CLIPViT, Attack Type=White-box2026.04 | 100 | — | — | |
| HQA-VLAttackSurrogate Model=CLIPCNN, Victim Model=CLIPCNN, Attack Type=White-box2026.04 | 99.87 | — | — | |
| HQA-VLAttackSurrogate Model=ALBEF, Victim Model=ALBEF, Attack Type=White-box2026.04 | 99.79 | — | — | |
| HQA-VLAttackSurrogate Model=TCL, Victim Model=TCL, Attack Type=White-box2026.04 | 99.79 | — | — | |
| BLIP-BASE# Pre-train Images=129M, Model scale=Base2023.05 | 97.3 | — | — | |
| SGASurrogate Model=ALBEF, Victim Model=ALBEF, Attack Type=White-box2026.04 | 97.24 | — | — | |
| DRASurrogate Model=ALBEF, Victim Model=ALBEF, Attack Type=White-box2026.04 | 96.14 | — | — | |
| ALBEF-BASE (14M)# Pre-train Images=14M, Model scale=Base2023.05 | 95.9 | — | — | |
| ManagerTower-BASE# Pre-train Images=4M, Model scale=Base2023.05 | 95.64 | — | — | |
| EVA-CLIPParameters=18B, Zero-shot=true2024.12 | 95.3 | — | 99.8 | |
| BridgeTower-BASE# Pre-train Images=4M, Model scale=Base2023.05 | 94.73 | — | — | |
| EVA-CLIPParameters=8B, Zero-shot=true2024.12 | 94.5 | — | 99.7 | |
| ALBEF-BASE# Pre-train Images=4M, Model scale=Base2023.05 | 94.3 | — | — | |
| METER-CLIP-BASE# Pre-train Images=4M, Model scale=Base2023.05 | 94.3 | — | — | |
| VladVABackbone=LLAVA-1.5-7B, Zero-shot=true2024.12 | 94.3 | — | 99.9 | |
| EVA-02-CLIPBackbone=ViT-E/14+, Zero-shot=true2024.12 | 93.9 | — | 99.8 | |
| OpenCLIPBackbone=ViT-BigG/14, Zero-shot=true2024.12 | 92.9 | — | 97.1 | |
| METER-Swin-BASE# Pre-train Images=4M, Model scale=Base2023.05 | 92.4 | — | — | |
| VLMo-BASE# Pre-train Images=4M, Model scale=Base2023.05 | 92.3 | — | — | |
| OpenCLIPBackbone=ViT-G/14, Zero-shot=true2024.12 | 91.5 | — | 99.6 | |
| VLM2VecBackbone=Mistral-7B, Zero-shot=true2024.12 | 90.3 | — | 99.6 | |
| UNIMO-BASE# Pre-train Images=4M, Model scale=Base2023.05 | 89.7 | — | — | |
| UNIMO-LARGE# Pre-train Images=4M, Model scale=Large2023.05 | 89.4 | — | — | |
| Flamingo-3Bparameters=3.2B, task-specific fine-tuning=false2022.11 | 89.3 | — | — | |
| Uni-Perceiver-v2 LARGEparameters=446M, task-specific fine-tuning=false2022.11 | 89.3 | — | — | |
| E5-VBackbone=LLAVA-Next-8B, Zero-shot=true2024.12 | 88.2 | — | 99.4 | |
| Uni-Perceiver-v2 BASEparameters=308M, task-specific fine-tuning=false2022.11 | 88.1 | — | — | |
| SuperCLIPPretrain=L-12.8B, Zero-shot=true2025.12 | 88.1 | — | — | |
| Uni-Perceivertuning_setting=fine-tuning, data_percentage=100%2021.12 | 87.9 | 98.2 | 99.1 | |
| VLM2VecTrain Data=1M, MLLM=Qwen2-VL 7B2025.09 | 87.4 | 97.7 | 99.1 | |
| UNITER-LARGE# Pre-train Images=4M, Model scale=Large2023.05 | 87.3 | — | — | |
| CLIPBackbone=ViT-L, Zero-shot=true2024.12 | 87.2 | — | 99.4 | |
| CLIPPretrain=L-12.8B, Zero-shot=true2025.12 | 87 | — | — | |
| E5-VBackbone=LLAVA-1.5-7B, Zero-shot=true2024.12 | 86.6 | — | 99 | |
| Uni-Perceivertuning_setting=prompt-tuning, data_percentage=10%2021.12 | 86.4 | 98.2 | 99.5 | |
| Unicoder-VLtuning_setting=fine-tuning2021.12 | 86.2 | 96.3 | 99 | |
| BLIP2Backbone=ViT-L, Zero-shot=true2024.12 | 86.1 | — | 99.4 | |
| UNITER-Btuning_setting=fine-tuning2021.12 | 85.9 | 97.1 | 98.8 | |
| UNITER-BASE# Pre-train Images=4M, Model scale=Base2023.05 | 85.9 | — | — | |
| Uni-Perceivertuning_setting=fine-tuning, data_percentage=10%2021.12 | 84.9 | 97.4 | 98.3 | |
| Uni-Perceivertuning_setting=prompt-tuning, data_percentage=1%2021.12 | 84.4 | 97.8 | 99.2 | |
| Uni-Perceiver LARGEparameters=354M, task-specific fine-tuning=false2022.11 | 83.7 | — | — | |
| Uni-Perceiver-MOE LARGEparameters=505M, task-specific fine-tuning=false2022.11 | 83.6 | — | — | |
| ViLTtuning_setting=fine-tuning2021.12 | 83.5 | 96.7 | 98.6 | |
| ViLT-BASE# Pre-train Images=4M, Model scale=Base2023.05 | 83.5 | — | — | |
| Uni-Perceiver BASEparameters=124M, task-specific fine-tuning=false2022.11 | 82.3 | — | — | |
| Uni-Perceiver-MOE BASEparameters=167M, task-specific fine-tuning=false2022.11 | 82.1 | — | — | |
| FreeRet_{r20}Train Data=-, MLLM=Qwen2.5-VL 7B2025.09 | 80.8 | 96.8 | 97.9 | |
| UNITER-Btuning_setting=without tuning2021.12 | 80.7 | 95.7 | 98 | |
| FreeRet_{r20}Train Data=-, MLLM=Qwen2.5-VL 3B2025.09 | 79.9 | 94.3 | 95.7 | |
| SuperCLIPPretrain=L-512M, Zero-shot=true2025.12 | 79.3 | — | — | |
| Uni-Perceivertuning_setting=fine-tuning, data_percentage=1%2021.12 | 78.4 | 95.7 | 97.8 | |
| UNIMOModel Size=large2020.12 | 78.04 | 94.24 | 97.12 | |
| Co-AttackSurrogate Model=ALBEF, Victim Model=ALBEF, Attack Type=White-box2026.04 | 77.16 | — | — | |
| ERNIE-ViLModel Size=large2020.12 | 76.7 | 93.58 | 96.44 | |
| CLIPPretrain=L-512M, Zero-shot=true2025.12 | 76.4 | — | — | |
| VillaModel Size=large2020.12 | 76.26 | 94.24 | 96.84 | |
| SuperCLIPPretrain=B-512M, Zero-shot=true2025.12 | 75.6 | — | — | |
| UNITERModel Size=large2020.12 | 75.56 | 94.08 | 96.76 | |
| BLIPBackbone=ViT-L, Zero-shot=true2024.12 | 75.5 | — | 97.7 | |
| Uni-Perceivertuning_setting=without tuning2021.12 | 74.8 | 94.8 | 98.2 | |
| VillaModel Size=base2020.12 | 74.74 | 92.86 | 95.82 | |
| UNIMOModel Size=base2020.12 | 74.66 | 93.4 | 96.08 | |
| ERNIE-ViLModel Size=base2020.12 | 74.44 | 92.72 | 95.94 | |
| CLIPPretrain=B-512M, Zero-shot=true2025.12 | 73.3 | — | — | |
| ViLTtuning_setting=without tuning2021.12 | 73.2 | 93.6 | 96.5 | |
| HQA-VLAttackSurrogate Model=ALBEF, Victim Model=TCL, Attack Type=Black-box2026.04 | 73.02 | — | — | |
| UNITERModel Size=base2020.12 | 72.52 | 92.36 | 96.08 | |
| ImageBERTtuning_setting=without tuning2021.12 | 70.7 | 90.2 | 94 | |
| FreeRet-embedTrain Data=-, MLLM=Qwen2.5-VL 3B2025.09 | 67.4 | 88.8 | 93.2 | |
| Sep-AttackSurrogate Model=ALBEF, Victim Model=ALBEF, Attack Type=White-box2026.04 | 65.69 | — | — | |
| LLaVA-1.5-7BZero-shot=true2024.12 | 65.6 | — | 92.3 | |
| LXMERT-SZero-shot=false2021.07 | 64.6 | 87.5 | 90.4 | |
| HQA-VLAttackSurrogate Model=TCL, Victim Model=ALBEF, Attack Type=Black-box2026.04 | 62.88 | — | — | |
| E5VTrain Data=-, MLLM=Qwen2.5-VL 7B2025.09 | 60.3 | 82.9 | 89.9 | |
| CLIP-PGSImage Enc.=ViT-B/16, Ratio=0.3, Zero-shot=true2025.03 | 59.9 | 83.5 | 90.8 | |
| HQA-VLAttackSurrogate Model=ALBEF, Victim Model=CLIPCNN, Attack Type=Black-box2026.04 | 59.64 | — | — | |
| E5VTrain Data=-, MLLM=Qwen2.5-VL 3B2025.09 | 59.3 | 81.3 | 88.4 | |
| CLIPImage Enc.=ViT-B/16, Zero-shot=true2025.03 | 58.5 | 83.8 | 89.1 | |
| ViLBERTModel Size=base2020.12 | 58.2 | 84.9 | 91.52 | |
| CLIP-PGSImage Enc.=ViT-B/16, Ratio=0.5, Zero-shot=true2025.03 | 57.7 | 82.7 | 90.4 | |
| CLIP-PGSImage Enc.=ViT-S/16, Ratio=0.3, Zero-shot=true2025.03 | 55.9 | 80.5 | 88.1 | |
| E-CLIPImage Enc.=ViT-B/16, Zero-shot=true2025.03 | 55.8 | 84.2 | 89.6 | |
| A-CLIPImage Enc.=ViT-B/16, Zero-shot=true2025.03 | 55.3 | 81.4 | 87.6 | |
| CLIP-PGSImage Enc.=ViT-S/16, Ratio=0.5, Zero-shot=true2025.03 | 53.9 | 80 | 87.4 | |
| FLIPImage Enc.=ViT-B/16, Zero-shot=true2025.03 | 53.8 | 80.8 | 88.5 | |
| PGDSurrogate Model=ALBEF, Victim Model=ALBEF, Attack Type=White-box2026.04 | 52.45 | — | — | |
| HQA-VLAttackSurrogate Model=ALBEF, Victim Model=CLIPViT, Attack Type=Black-box2026.04 | 52.15 | — | — | |
| LXMERT-SZero-shot=true2021.07 | 49 | 75 | 81.8 | |
| CyCLIPsmode=zero-shot2023.10 | 40 | 69.3 | 79.7 | |
| CyCLIPmode=zero-shot2023.10 | 37.3 | 66.1 | 76.4 | |
| CLIPsmode=zero-shot2023.10 | 35.2 | 63.2 | 75.3 | |
| CLIPmode=zero-shot2023.10 | 31.8 | 62.1 | 72.9 | |
| PTM-STNumber of distilled pairs=10002026.03 | 23.8 | 53.6 | 67.2 | |
| LXMERTZero-shot=true2021.07 | 23.6 | 51.5 | 61.3 | |
| PTM-STPairs=500, Ratio=1.7%2026.03 | 22.2 | 51.1 | 64.6 | |
| PTM-STPairs=200, Ratio=0.7%2026.03 | 19.3 | 45.9 | 58.9 | |
| LoRSNumber of distilled pairs=10002026.03 | 14.9 | 39.8 | 53.5 | |
| LoRSPairs=500, Ratio=1.7%2026.03 | 14.7 | 37.6 | 51.1 | |
| EDGENumber of distilled pairs=10002026.03 | 14.5 | 38.3 | 51.7 |