1,000-pair benchmark
Benchmarks
TaskDatasetSOTA resultTrendResults
1,000-pair benchmark (test)
59.9Attack Success Rate (ASR)
12
1,000-pair benchmark Kimi-K2.5
51.2ASR
8
1,000-pair benchmark Gemma-3-27B-it
24.6ASR
8
1,000-pair benchmark Llama-3.2-11B-V
38Attack Success Rate (ASR)
8
1,000-pair benchmark GLM-4.6V
74.9Attack Success Rate (ASR)
8
1,000-pair benchmark Qwen3-VL-32B
60.1ASR
8
1,000-pair benchmark Qwen3-VL-8B
67ASR (%)
8