GUI-based Adversarial Attack on EVA-GUI 1.0 (test)
96.3Attack Success Rate (Amazon)EVA
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| EVAVictim Agent=Qwen3-VL-8B-Instruct2025.05 | 96.3 | 91.53 | 84.41 | 66.67 | 84.73 | |
| EVAVictim Agent=GPT-4-Vision-Preview2025.05 | 94.71 | 70.37 | 18.52 | 61.9 | 61.38 | |
| EVAVictim Agent=Qwen2.5-VL-7B-Instruct2025.05 | 83.07 | 70.37 | 59.79 | 60.32 | 68.39 | |
| PopupAttackVictim Agent=Qwen3-VL-8B-Instruct2025.05 | 81.72 | 51.85 | 36.51 | 65.59 | 58.92 | |
| EVAVictim Agent=UI-TARS-1.5-7B2025.05 | 70.37 | 59.79 | 47.62 | 60.32 | 59.53 | |
| Direct-LLMVictim Agent=Qwen3-VL-8B-Instruct2025.05 | 69.84 | 66.14 | 45.5 | 36.51 | 54.5 | |
| Direct-LLMVictim Agent=GPT-4-Vision-Preview2025.05 | 68.25 | 53.44 | 29.1 | 26.98 | 44.44 | |
| EVAVictim Agent=GUI-Owl-7B2025.05 | 59.26 | 48.68 | 59.79 | 69.31 | 59.26 | |
| Direct-LLMVictim Agent=UI-TARS-1.5-7B2025.05 | 40.74 | 38.62 | 44.44 | 43.92 | 41.93 | |
| Direct-LLMVictim Agent=Qwen2.5-VL-7B-Instruct2025.05 | 37.63 | 41.8 | 35.45 | 40.21 | 38.77 | |
| PopupAttackVictim Agent=GPT-4-Vision-Preview2025.05 | 30.69 | 46.24 | 49.74 | 49.18 | 43.96 | |
| PopupAttackVictim Agent=UI-TARS-1.5-7B2025.05 | 29.8 | 53.5 | 37.1 | 49 | 42.35 | |
| Direct-LLMVictim Agent=GUI-Owl-7B2025.05 | 26.98 | 28.04 | 29.1 | 51.85 | 33.99 | |
| SeedVictim Agent=GPT-4-Vision-Preview2025.05 | 24.31 | 22.92 | 26.39 | 20.83 | 23.61 | |
| SeedVictim Agent=UI-TARS-1.5-7B2025.05 | 23.54 | 24.21 | 30.09 | 24.47 | 25.58 | |
| SeedVictim Agent=Qwen3-VL-8B-Instruct2025.05 | 23.08 | 13.36 | 28.24 | 16.87 | 20.39 | |
| SeedVictim Agent=Qwen2.5-VL-7B-Instruct2025.05 | 16.14 | 9.26 | 25.73 | 13.56 | 16.17 | |
| PopupAttackVictim Agent=GUI-Owl-7B2025.05 | 13.8 | 56.5 | 42.9 | 68.3 | 45.33 | |
| PopupAttackVictim Agent=Qwen2.5-VL-7B-Instruct2025.05 | 9.29 | 22.75 | 30.6 | 44.44 | 26.77 | |
| SeedVictim Agent=GUI-Owl-7B2025.05 | 0 | 0 | 12.63 | 3.84 | 4.12 |