GUI Agent Robustness Evaluation on Pop-up Perturbation Avg. 1.0
96.3DSRCA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| CABackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Base Defense Method=CA, Injection Type (IT)=Overlay2025.07 | 96.3 | 100 | 3.7 | |
| CABackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Base Defense Method=CA, Injection Type (IT)=Inductive2025.07 | 92.7 | 99.8 | 7.1 | |
| CABackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Base Defense Method=CA, Injection Type (IT)=Overlay2025.07 | 84.7 | 86.9 | 2.2 | |
| CABackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Base Defense Method=CA, Injection Type (IT)=Inductive2025.07 | 75.2 | 90.3 | 15.1 | |
| NDBackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Base Defense Method=ND, Injection Type (IT)=Overlay2025.07 | 68.6 | 81.2 | 12.6 | |
| NDBackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Base Defense Method=ND, Injection Type (IT)=Inductive2025.07 | 60.8 | 78 | 17.2 | |
| DPOBackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Base Defense Method=DPO, Injection Type (IT)=Overlay2025.07 | 52.3 | 75.4 | 23.1 | |
| DPOBackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Base Defense Method=DPO, Injection Type (IT)=Inductive2025.07 | 44.9 | 72.2 | 27.3 | |
| DABackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Base Defense Method=DA, Injection Type (IT)=Inductive2025.07 | 42.8 | 79.8 | 37 | |
| DABackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Base Defense Method=DA, Injection Type (IT)=Overlay2025.07 | 39.5 | 64.8 | 25.3 | |
| DABackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Base Defense Method=DA, Injection Type (IT)=Inductive2025.07 | 38.4 | 78.8 | 40.4 | |
| DABackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Base Defense Method=DA, Injection Type (IT)=Overlay2025.07 | 28.7 | 78.3 | 49.6 | |
| NDBackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Base Defense Method=ND, Injection Type (IT)=Overlay2025.07 | 18.9 | 66.4 | 47.5 | |
| DPOBackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Base Defense Method=DPO, Injection Type (IT)=Overlay2025.07 | 15.9 | 63.9 | 48 | |
| NDBackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Base Defense Method=ND, Injection Type (IT)=Inductive2025.07 | 14.8 | 68.3 | 53.5 | |
| DPOBackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Base Defense Method=DPO, Injection Type (IT)=Inductive2025.07 | 10.5 | 65.2 | 54.7 |