Mobile GUI Automation on PrivScreen
93AccuracyOriginal
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| OriginalModel=GPT-52026.02 | 93 | 97.14 | 97.15 | 0.926 | 0.9627 | 0.8584 | 0.9246 | |
| DualTAPModel=GPT-52026.02 | 91 | 23.33 | 24.43 | 0.1589 | 0.2379 | 0.0941 | 0.1628 | |
| AnyAttackModel=GPT-52026.02 | 90 | 91.43 | 87.49 | 0.6757 | 0.849 | 0.4978 | 0.7446 | |
| OursModel=GPT-52026.02 | 90 | 18.84 | 26.74 | 0.1495 | 0.3224 | 0.1587 | 0.1892 | |
| OriginalModel=Qwen2.5-VL-7B2026.02 | 89 | 97.14 | 96.99 | 0.8675 | 0.9011 | 0.783 | 0.8991 | |
| OursModel=Qwen2.5-VL-7B2026.02 | 89 | 19.52 | 30.69 | 0.1411 | 0.3966 | 0.1496 | 0.1832 | |
| DualTAPModel=Qwen2.5-VL-7B2026.02 | 88 | 32.86 | 38.82 | 0.1998 | 0.4202 | 0.0908 | 0.166 | |
| VIPModel=GPT-52026.02 | 86 | 80 | 79.33 | 0.6662 | 0.7715 | 0.5284 | 0.6846 | |
| VIPModel=Qwen2.5-VL-7B2026.02 | 81 | 94.76 | 92.67 | 0.7556 | 0.896 | 0.5932 | 0.7856 | |
| FOA-AttackModel=GPT-52026.02 | 80 | 80.95 | 79.72 | 0.6293 | 0.7532 | 0.4967 | 0.7004 | |
| OriginalModel=UI-TARS-7B2026.02 | 80 | 93.81 | 93.1 | 0.7846 | 0.9079 | 0.6721 | 0.8363 | |
| AnyAttackModel=UI-TARS-7B2026.02 | 76 | 81.43 | 80.11 | 0.5533 | 0.7615 | 0.3748 | 0.6057 | |
| FOA-AttackModel=Qwen2.5-VL-7B2026.02 | 73 | 83.81 | 82.84 | 0.6307 | 0.815 | 0.5343 | 0.7309 | |
| AnyAttackModel=Qwen2.5-VL-7B2026.02 | 70 | 95.24 | 91.86 | 0.6721 | 0.8705 | 0.5164 | 0.7317 | |
| DualTAPModel=UI-TARS-7B2026.02 | 67 | 34.76 | 38.35 | 0.2278 | 0.4535 | 0.1139 | 0.2034 | |
| VIPModel=UI-TARS-7B2026.02 | 66 | 89.05 | 85.83 | 0.6739 | 0.8288 | 0.5232 | 0.7223 | |
| FOA-AttackModel=UI-TARS-7B2026.02 | 64 | 76.67 | 75.6 | 0.5349 | 0.7187 | 0.4229 | 0.6178 | |
| OursModel=UI-TARS-7B2026.02 | 61 | 18.1 | 29.15 | 0.1251 | 0.3695 | 0.1029 | 0.1552 |