GUI Agent Robustness Evaluation on Pop-up Perturbation 1.0 (Small)
97DSRCA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| CABackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Overlay, Visual Prompting=Highlight2025.07 | 97 | 100 | 3 | |
| CABackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Overlay, Visual Prompting=Default2025.07 | 97 | 87.4 | 9.6 | |
| CABackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Overlay, Visual Prompting=Default2025.07 | 96.5 | 100 | 3.5 | |
| CABackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Overlay, Visual Prompting=Highlight2025.07 | 94 | 85.4 | 8.6 | |
| CABackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Inductive, Visual Prompting=Highlight2025.07 | 93.5 | 100 | 6.5 | |
| CABackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Inductive, Visual Prompting=Default2025.07 | 92.5 | 100 | 7.5 | |
| CABackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Inductive, Visual Prompting=Highlight2025.07 | 87 | 91.5 | 4.5 | |
| CABackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Inductive, Visual Prompting=Default2025.07 | 85.5 | 90.5 | 5 | |
| NDBackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Overlay, Visual Prompting=Default2025.07 | 64.3 | 81.4 | 17.1 | |
| NDBackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Inductive, Visual Prompting=Highlight2025.07 | 61 | 77 | 16 | |
| NDBackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Inductive, Visual Prompting=Default2025.07 | 59.5 | 76.5 | 17 | |
| NDBackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Overlay, Visual Prompting=Highlight2025.07 | 58.8 | 79.9 | 21.1 | |
| DPOBackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Inductive, Visual Prompting=Highlight2025.07 | 46 | 74 | 28 | |
| DPOBackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Inductive, Visual Prompting=Default2025.07 | 45.5 | 72 | 26.5 | |
| DPOBackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Overlay, Visual Prompting=Highlight2025.07 | 43.7 | 74.4 | 30.7 | |
| DPOBackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Overlay, Visual Prompting=Default2025.07 | 42.2 | 72.4 | 30.2 | |
| DABackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Inductive, Visual Prompting=Default2025.07 | 41.5 | 75.5 | 34 | |
| DABackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Inductive, Visual Prompting=Highlight2025.07 | 41.5 | 78.5 | 37 | |
| DABackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Overlay, Visual Prompting=Default2025.07 | 41.2 | 60.3 | 19.1 | |
| DABackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Overlay, Visual Prompting=Highlight2025.07 | 37.7 | 64.8 | 27.1 | |
| DABackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Inductive, Visual Prompting=Default2025.07 | 32.5 | 73.5 | 41 | |
| DABackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Inductive, Visual Prompting=Highlight2025.07 | 32.5 | 75 | 42.5 | |
| NDBackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Overlay, Visual Prompting=Highlight2025.07 | 25.1 | 64.3 | 39.2 | |
| DABackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Overlay, Visual Prompting=Default2025.07 | 21.6 | 74.4 | 52.8 | |
| DABackbone=LLaVA-v1.6-Vicuna-13B (LaSM L12-28, α=1.2), Injection Type (IT)=Overlay, Visual Prompting=Highlight2025.07 | 21.6 | 74.4 | 52.8 | |
| NDBackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Inductive, Visual Prompting=Highlight2025.07 | 21 | 67 | 46 | |
| NDBackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Overlay, Visual Prompting=Default2025.07 | 20.6 | 65.8 | 45.2 | |
| DPOBackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Overlay, Visual Prompting=Highlight2025.07 | 20.6 | 61.8 | 41.2 | |
| NDBackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Inductive, Visual Prompting=Default2025.07 | 19.5 | 67 | 47.5 | |
| DPOBackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Overlay, Visual Prompting=Default2025.07 | 18.1 | 65.8 | 47.7 | |
| DPOBackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Inductive, Visual Prompting=Highlight2025.07 | 15.5 | 63.5 | 48 | |
| DPOBackbone=Qwen2-vl-7B (LaSM L7-18, α=1.1), Injection Type (IT)=Inductive, Visual Prompting=Default2025.07 | 15 | 65 | 50 |