Generation on ViDoRe AI v1 (test)
1ASR-G Similarity (mean) @-1MO-PGD
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| MO-PGDAttack Type=White-box, Embedder=CLIP-L, VLM=Qwen2.5-3B2025.04 | 1 | 1 | 0.03 | 0.51 | |
| MO-PGDAttack Type=White-box, Embedder=CLIP-L, VLM=SmolVLM2025.04 | 1 | 1 | 0.03 | 0.5 | |
| MO-PGDAttack Type=White-box, Embedder=GME, VLM=InternVL3-2B2025.04 | 1 | 1 | 0.03 | 0.52 | |
| MO-PGDAttack Type=White-box, Embedder=GME, VLM=Qwen2.5-3B2025.04 | 1 | 1 | 0.03 | 0.52 | |
| MO-PGDAttack Type=White-box, Embedder=GME, VLM=SmolVLM2025.04 | 0.99 | 1 | 0.03 | 0.5 | |
| MO-PGDAttack Type=White-box, Embedder=ColPali, VLM=Qwen2.5-3B2025.04 | 0.97 | 1 | 0.04 | 0.51 | |
| MO-PGDAttack Type=White-box, Embedder=CLIP-L, VLM=InternVL3-2B2025.04 | 0.96 | 1 | 0.04 | 0.51 | |
| MO-PGDAttack Type=Component-wise Transfer, Embedder=Any, VLM=Same2025.04 | 0.91 | 1 | 0.07 | 0.47 | |
| MO-PGDAttack Type=In-set Model Ensemble, Embedder=ColPali, VLM=Qwen2.5-3B2025.04 | 0.9 | 1 | 0.07 | 0.46 | |
| MO-PGDAttack Type=In-set Model Ensemble, Embedder=GME, VLM=Qwen2.5-3B2025.04 | 0.89 | 1 | 0.08 | 0.47 | |
| MO-PGDAttack Type=In-set Model Ensemble, Embedder=CLIP-L, VLM=Qwen2.5-3B2025.04 | 0.88 | 1 | 0.09 | 0.46 | |
| MO-PGDAttack Type=White-box, Embedder=ColPali, VLM=SmolVLM2025.04 | 0.87 | 1 | 0.06 | 0.47 | |
| MO-PGDAttack Type=White-box, Embedder=ColPali, VLM=InternVL3-2B2025.04 | 0.44 | 0.89 | 0.3 | 0.25 | |
| MO-PGDAttack Type=In-set Model Ensemble, Embedder=ColPali, VLM=InternVL3-2B2025.04 | 0.14 | 0.69 | 0.47 | 0.09 | |
| MO-PGDAttack Type=In-set Model Ensemble, Embedder=GME, VLM=InternVL3-2B2025.04 | 0.13 | 0.7 | 0.46 | 0.09 | |
| MO-PGDAttack Type=In-set Model Ensemble, Embedder=CLIP-L, VLM=InternVL3-2B2025.04 | 0.12 | 0.58 | 0.48 | 0.07 | |
| MO-PGDAttack Type=In-set Model Ensemble, Embedder=GME, VLM=SmolVLM2025.04 | 0.12 | 0.32 | 0.46 | 0.08 | |
| MO-PGDAttack Type=In-set Model Ensemble, Embedder=CLIP-L, VLM=SmolVLM2025.04 | 0.09 | 0.4 | 0.45 | 0.09 | |
| MO-PGDAttack Type=In-set Model Ensemble, Embedder=ColPali, VLM=SmolVLM2025.04 | 0.07 | 0.33 | 0.47 | 0.08 | |
| MO-PGDAttack Type=Out-set Model Ensemble, Embedder=Any, VLM=Any2025.04 | 0.01 | 0.03 | 0.54 | 0 | |
| MO-PGDAttack Type=Complete Transfer, Embedder=Any, VLM=Any2025.04 | 0.01 | 0.04 | 0.54 | 0 | |
| MO-PGDAttack Type=Component-wise Transfer, Embedder=Same, VLM=Any2025.04 | 0.01 | 0.03 | 0.54 | 0 | |
| MO-PGDAttack Type=Prompt-based (Any), Embedder=Any, VLM=Any2025.04 | 0.01 | 0.1 | 0.52 | — |