Visual Jailbreaking Attack on AdvBench (ASR, Query)
43.84ASRFORCE
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| FORCEArchitecture=Adapter-Based MLLMs, Target Model=Llava-v1.6-mistral-7b2025.09 | 43.84 | 59.76 | |
| FORCEArchitecture=Adapter-Based MLLMs, Target Model=Idefics3-8B-Llama32025.09 | 35.96 | 67.49 | |
| PGDArchitecture=Adapter-Based MLLMs, Target Model=Llava-v1.6-mistral-7b2025.09 | 35.19 | 67.82 | |
| PGDArchitecture=Adapter-Based MLLMs, Target Model=Idefics3-8B-Llama32025.09 | 29.81 | 71.57 | |
| FORCEArchitecture=Adapter-Based MLLMs, Target Model=InstructBlip-Vicuna-7B2025.09 | 27.88 | 77.04 | |
| PGDArchitecture=Adapter-Based MLLMs, Target Model=InstructBlip-Vicuna-7B2025.09 | 25.58 | 79.45 | |
| FORCEArchitecture=Commercial MLLMs, Target Model=Gemini-2.5-Pro2025.09 | 6 | 95.17 | |
| PGDArchitecture=Commercial MLLMs, Target Model=Gemini-2.5-Pro2025.09 | 4 | 96.59 | |
| FORCEArchitecture=Early-Fusion MLLMs, Target Model=Qwen2.5-VL-7B-Instruct2025.09 | 2.69 | 97.22 | |
| FORCEArchitecture=Early-Fusion MLLMs, Target Model=Llama-3.2-11B-Vision-Instruct2025.09 | 2.31 | 98.02 | |
| PGDArchitecture=Early-Fusion MLLMs, Target Model=Qwen2.5-VL-7B-Instruct2025.09 | 1.54 | 98.65 | |
| PGDArchitecture=Early-Fusion MLLMs, Target Model=Llama-3.2-11B-Vision-Instruct2025.09 | 1.15 | 98.94 | |
| PGDArchitecture=Commercial MLLMs, Target Model=Claude-Sonnet-42025.09 | 1 | 99.91 | |
| FORCEArchitecture=Commercial MLLMs, Target Model=Claude-Sonnet-42025.09 | 1 | 99.22 | |
| FORCEArchitecture=Commercial MLLMs, Target Model=GPT-52025.09 | 1 | 99.05 | |
| FORCEArchitecture=Adapter-Based MLLMs, Target Model=Llava-v1.6-mistral-7b2025.12 | 0.4384 | 59.76 | |
| FORCEArchitecture=Adapter-Based MLLMs, Target Model=Idefics3-8B-Llama32025.12 | 0.3596 | 67.49 | |
| PGDArchitecture=Adapter-Based MLLMs, Target Model=Llava-v1.6-mistral-7b2025.12 | 0.3519 | 67.82 | |
| PGDArchitecture=Adapter-Based MLLMs, Target Model=Idefics3-8B-Llama32025.12 | 0.2981 | 71.57 | |
| FORCEArchitecture=Adapter-Based MLLMs, Target Model=InstructBlip-Vicuna-7B2025.12 | 0.2788 | 77.04 | |
| PGDArchitecture=Adapter-Based MLLMs, Target Model=InstructBlip-Vicuna-7B2025.12 | 0.2558 | 79.45 | |
| FORCEArchitecture=Commercial MLLMs, Target Model=Gemini-2.5-Pro2025.12 | 0.06 | 95.17 | |
| PGDArchitecture=Commercial MLLMs, Target Model=Gemini-2.5-Pro2025.12 | 0.04 | 96.59 | |
| FORCEArchitecture=Early-Fusion MLLMs, Target Model=Qwen2.5-VL-7B-Instruct2025.12 | 0.0269 | 97.22 | |
| FORCEArchitecture=Early-Fusion MLLMs, Target Model=Llama-3.2-11B-Vision-Instruct2025.12 | 0.0231 | 98.02 | |
| PGDArchitecture=Early-Fusion MLLMs, Target Model=Qwen2.5-VL-7B-Instruct2025.12 | 0.0154 | 98.65 | |
| PGDArchitecture=Early-Fusion MLLMs, Target Model=Llama-3.2-11B-Vision-Instruct2025.12 | 0.0115 | 98.94 | |
| PGDArchitecture=Commercial MLLMs, Target Model=Claude-Sonnet-42025.12 | 0.01 | 99.91 | |
| FORCEArchitecture=Commercial MLLMs, Target Model=Claude-Sonnet-42025.12 | 0.01 | 99.22 | |
| FORCEArchitecture=Commercial MLLMs, Target Model=GPT-52025.12 | 0.01 | 99.05 | |
| PGDArchitecture=Commercial MLLMs, Target Model=GPT-52025.12 | 0 | 100 | |
| PGDArchitecture=Commercial MLLMs, Target Model=GPT-52025.09 | 0 | 100 |