GUI Agent Robustness Evaluation on Pop-up Perturbation 1.0 (Large)
97.5DSR (%)CA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| CABackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Overlay, Visual Prompting=Highlight2025.07 | 97.5 | 100 | 2.5 | |
| CABackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Overlay, Visual Prompting=Default2025.07 | 97 | 100 | 3 | |
| CABackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Inductive, Visual Prompting=Highlight2025.07 | 91.5 | 99.5 | 8 | |
| CABackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Inductive, Visual Prompting=Default2025.07 | 89 | 99 | 10 | |
| NDBackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Overlay, Visual Prompting=Highlight2025.07 | 75.4 | 80.9 | 5.5 | |
| NDBackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Overlay, Visual Prompting=Default2025.07 | 70.9 | 80 | 9.1 | |
| CABackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Overlay, Visual Prompting=Highlight2025.07 | 70.9 | 86.9 | 16 | |
| CABackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Overlay, Visual Prompting=Default2025.07 | 67.8 | 88.4 | 20.6 | |
| CABackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Inductive, Visual Prompting=Highlight2025.07 | 67 | 84 | 17 | |
| NDBackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Inductive, Visual Prompting=Highlight2025.07 | 61.5 | 77 | 15.5 | |
| CABackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Inductive, Visual Prompting=Default2025.07 | 61.5 | 92.5 | 31 | |
| DPOBackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Overlay, Visual Prompting=Highlight2025.07 | 57.3 | 76.9 | 19.6 | |
| DPOBackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Overlay, Visual Prompting=Default2025.07 | 56.3 | 74.4 | 18.1 | |
| NDBackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Inductive, Visual Prompting=Default2025.07 | 56 | 73 | 17 | |
| DPOBackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Inductive, Visual Prompting=Highlight2025.07 | 44.5 | 72 | 27.5 | |
| DABackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Inductive, Visual Prompting=Highlight2025.07 | 44 | 81 | 37 | |
| DABackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Inductive, Visual Prompting=Default2025.07 | 42.5 | 80.5 | 38 | |
| DABackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Inductive, Visual Prompting=Highlight2025.07 | 42.5 | 82.5 | 40 | |
| DPOBackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Inductive, Visual Prompting=Default2025.07 | 41 | 68.5 | 27.5 | |
| DABackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Inductive, Visual Prompting=Default2025.07 | 40.5 | 82.5 | 42 | |
| DABackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Overlay, Visual Prompting=Highlight2025.07 | 36.7 | 65.8 | 29.1 | |
| DABackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Overlay, Visual Prompting=Default2025.07 | 36.2 | 65.8 | 29.6 | |
| DABackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Overlay, Visual Prompting=Highlight2025.07 | 32.2 | 79.5 | 47.3 | |
| DABackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Overlay, Visual Prompting=Default2025.07 | 31.7 | 84.4 | 52.7 | |
| NDBackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Overlay, Visual Prompting=Highlight2025.07 | 14.1 | 64.3 | 50.2 | |
| NDBackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Overlay, Visual Prompting=Default2025.07 | 13.1 | 62.3 | 49.2 | |
| DPOBackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Overlay, Visual Prompting=Highlight2025.07 | 11.6 | 63.8 | 52.2 | |
| NDBackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Inductive, Visual Prompting=Highlight2025.07 | 10.5 | 71.5 | 61 | |
| DPOBackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Overlay, Visual Prompting=Default2025.07 | 9.6 | 60.8 | 51.2 | |
| NDBackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Inductive, Visual Prompting=Default2025.07 | 9.5 | 65 | 55.5 | |
| DPOBackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Inductive, Visual Prompting=Highlight2025.07 | 6.5 | 68.5 | 62 | |
| DPOBackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Inductive, Visual Prompting=Default2025.07 | 6 | 62.5 | 56.5 |