Document Retrieval on ViDoRe AI v1 (test)
0.19Delta Recall@1MO-PGD
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| MO-PGDAttack Type=White-box, Embedder=CLIP-L, VLM=InternVL3-2B2025.04 | 0.19 | 21 | 97 | 44 | 0.01 | 100 | |
| MO-PGDAttack Type=White-box, Embedder=CLIP-L, VLM=Qwen2.5-3B2025.04 | 0.19 | 21 | 98 | 44 | 0.01 | 100 | |
| MO-PGDAttack Type=White-box, Embedder=CLIP-L, VLM=SmolVLM2025.04 | 0.17 | 21 | 90 | 44 | 0.01 | 99 | |
| MO-PGDAttack Type=Component-wise Transfer, Embedder=Same, VLM=Any2025.04 | 0.06 | — | 32 | — | 0 | 40 | |
| MO-PGDAttack Type=In-set Model Ensemble, Embedder=CLIP-L, VLM=InternVL3-2B2025.04 | 0.01 | 21 | 7 | 44 | 0 | 33 | |
| MO-PGDAttack Type=In-set Model Ensemble, Embedder=CLIP-L, VLM=Qwen2.5-3B2025.04 | 0.01 | 21 | 7 | 44 | 0 | 33 | |
| MO-PGDAttack Type=In-set Model Ensemble, Embedder=CLIP-L, VLM=SmolVLM2025.04 | 0.01 | 21 | 7 | 44 | 0 | 33 | |
| MO-PGDAttack Type=White-box, Embedder=ColPali, VLM=InternVL3-2B2025.04 | 0 | 67 | 0 | 98 | 0 | 5 | |
| MO-PGDAttack Type=White-box, Embedder=ColPali, VLM=Qwen2.5-3B2025.04 | 0 | 67 | 0 | 98 | 0 | 5 | |
| MO-PGDAttack Type=White-box, Embedder=ColPali, VLM=SmolVLM2025.04 | 0 | 67 | 0 | 98 | 0 | 6 | |
| MO-PGDAttack Type=White-box, Embedder=GME, VLM=InternVL3-2B2025.04 | 0 | 58 | 0 | 94 | 0 | 19 | |
| MO-PGDAttack Type=White-box, Embedder=GME, VLM=Qwen2.5-3B2025.04 | 0 | 58 | 0 | 94 | 0 | 17 | |
| MO-PGDAttack Type=White-box, Embedder=GME, VLM=SmolVLM2025.04 | 0 | 58 | 0 | 94 | 0 | 13 | |
| MO-PGDAttack Type=In-set Model Ensemble, Embedder=ColPali, VLM=InternVL3-2B2025.04 | 0 | 67 | 0 | 98 | 0 | 1 | |
| MO-PGDAttack Type=In-set Model Ensemble, Embedder=ColPali, VLM=Qwen2.5-3B2025.04 | 0 | 67 | 0 | 98 | 0 | 1 | |
| MO-PGDAttack Type=In-set Model Ensemble, Embedder=ColPali, VLM=SmolVLM2025.04 | 0 | 67 | 0 | 98 | 0 | 1 | |
| MO-PGDAttack Type=In-set Model Ensemble, Embedder=GME, VLM=InternVL3-2B2025.04 | 0 | 58 | 0 | 94 | 0 | 4 | |
| MO-PGDAttack Type=In-set Model Ensemble, Embedder=GME, VLM=Qwen2.5-3B2025.04 | 0 | 58 | 0 | 94 | 0 | 4 | |
| MO-PGDAttack Type=In-set Model Ensemble, Embedder=GME, VLM=SmolVLM2025.04 | 0 | 58 | 0 | 94 | 0 | 4 | |
| MO-PGDAttack Type=Out-set Model Ensemble, Embedder=Any, VLM=Any2025.04 | 0 | — | 0 | — | 0 | 0 | |
| MO-PGDAttack Type=Complete Transfer, Embedder=Any, VLM=Any2025.04 | 0 | — | 0 | — | 0 | 0 | |
| MO-PGDAttack Type=Component-wise Transfer, Embedder=Any, VLM=Same2025.04 | 0 | — | 0 | — | 0 | 0 | |
| MO-PGDAttack Type=Prompt-based (Any), Embedder=Any, VLM=Any2025.04 | 0 | — | 0 | — | 0 | 0 |