GUI Agent Robustness Evaluation on Pop-up Perturbation 1.0 (Medium)
97Direct Success Rate (DSR)CA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| CABackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Overlay, Visual Prompting=Default2025.07 | 97 | 100 | 3 | |
| CABackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Inductive, Visual Prompting=Highlight2025.07 | 96.5 | 100 | 3.5 | |
| CABackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Inductive, Visual Prompting=Default2025.07 | 93 | 100 | 7 | |
| CABackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Overlay, Visual Prompting=Highlight2025.07 | 92.5 | 100 | 7.5 | |
| CABackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Overlay, Visual Prompting=Highlight2025.07 | 92 | 85.4 | 6.6 | |
| CABackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Overlay, Visual Prompting=Default2025.07 | 86.4 | 84.9 | 1.5 | |
| CABackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Inductive, Visual Prompting=Highlight2025.07 | 78 | 94 | 16 | |
| CABackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Inductive, Visual Prompting=Default2025.07 | 72 | 93 | 21 | |
| NDBackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Overlay, Visual Prompting=Highlight2025.07 | 71.4 | 84.4 | 13 | |
| NDBackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Overlay, Visual Prompting=Default2025.07 | 70.9 | 80.4 | 9.5 | |
| NDBackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Inductive, Visual Prompting=Highlight2025.07 | 67.5 | 83 | 15.5 | |
| NDBackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Inductive, Visual Prompting=Default2025.07 | 59.5 | 78.5 | 19 | |
| DPOBackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Overlay, Visual Prompting=Highlight2025.07 | 57.3 | 78.9 | 21.6 | |
| DPOBackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Overlay, Visual Prompting=Default2025.07 | 56.8 | 75.4 | 18.6 | |
| DPOBackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Inductive, Visual Prompting=Highlight2025.07 | 47 | 75.5 | 28.5 | |
| DPOBackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Inductive, Visual Prompting=Default2025.07 | 45.5 | 73 | 27.5 | |
| DABackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Inductive, Visual Prompting=Highlight2025.07 | 45 | 79.5 | 34.5 | |
| DABackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Overlay, Visual Prompting=Default2025.07 | 43.2 | 65.8 | 22.6 | |
| DABackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Inductive, Visual Prompting=Default2025.07 | 42.5 | 83.5 | 41 | |
| DABackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Overlay, Visual Prompting=Highlight2025.07 | 41.7 | 66.3 | 24.6 | |
| DABackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Inductive, Visual Prompting=Highlight2025.07 | 41.5 | 80 | 38.5 | |
| DABackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Inductive, Visual Prompting=Default2025.07 | 41 | 79.5 | 38.5 | |
| DABackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Overlay, Visual Prompting=Default2025.07 | 32.7 | 78.4 | 45.7 | |
| DABackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Overlay, Visual Prompting=Highlight2025.07 | 32.7 | 78.4 | 45.7 | |
| NDBackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Overlay, Visual Prompting=Highlight2025.07 | 20.6 | 68.8 | 48.2 | |
| NDBackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Overlay, Visual Prompting=Default2025.07 | 20.1 | 72.9 | 52.8 | |
| DPOBackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Overlay, Visual Prompting=Default2025.07 | 17.6 | 65.8 | 48.2 | |
| DPOBackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Overlay, Visual Prompting=Highlight2025.07 | 17.6 | 65.3 | 47.7 | |
| NDBackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Inductive, Visual Prompting=Default2025.07 | 15 | 69.5 | 54.5 | |
| NDBackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Inductive, Visual Prompting=Highlight2025.07 | 13.5 | 69.5 | 56 | |
| DPOBackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Inductive, Visual Prompting=Default2025.07 | 10 | 65.5 | 55.5 | |
| DPOBackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Inductive, Visual Prompting=Highlight2025.07 | 10 | 66 | 56 |