Action Prediction on AndroidControl Low-level
97.6TMR (Click)ICLAttack
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ICLAttackModel=Qwen2-VL-7B2025.05 | 97.6 | 58.7 | 99.1 | 76.7 | 99.6 | 82.1 | 97.9 | 90.7 | 98 | 81.3 | 96.7 | 69.1 | |
| CleanModel=Qwen2-VL-2B2025.05 | 97.2 | 75.3 | 98.7 | 77.4 | 99.7 | 81.1 | 97.9 | 93.1 | 98 | 84 | 96.7 | 79.9 | |
| ICLAttackModel=Qwen2-VL-2B2025.05 | 97.2 | 75 | 99.5 | 78.5 | 98.9 | 81.6 | 96.9 | 90.2 | 98.3 | 82 | 96.4 | 79.1 | |
| CleanBackbone=OS-Atlas-Base-7B2025.05 | 97.1 | 68.4 | 98.3 | 76.4 | 99.5 | 75.8 | 99 | 93.1 | 98 | 87.3 | 97 | 75.8 | |
| ICLAttackBackbone=OS-Atlas-Base-7B2025.05 | 97.1 | 69.9 | 99.1 | 78.6 | 99.4 | 80.8 | 99.3 | 92.4 | 98.3 | 87 | 97 | 76.4 | |
| CleanModel=Qwen2-VL-7B2025.05 | 96.5 | 56.8 | 97.7 | 75.5 | 99.7 | 80.4 | 98.2 | 92.5 | 97.6 | 86.5 | 96.4 | 68.7 | |
| AgentGhostBackbone=OS-Atlas-Base-7B2025.05 | 96.4 | 70.6 | 97.7 | 75.1 | 99.6 | 82.3 | 99.1 | 91.6 | 98 | 83.3 | 96.1 | 76.4 | |
| AgentGhostModel=Qwen2-VL-7B2025.05 | 94.9 | 55.7 | 97.4 | 74 | 99.3 | 79.5 | 95.7 | 88.7 | 97 | 87.5 | 95.2 | 67 | |
| AgentGhostModel=Qwen2-VL-2B2025.05 | 93.9 | 72.8 | 94.4 | 76 | 99.5 | 85.9 | 93.8 | 86.6 | 97.4 | 80.8 | 93.6 | 77.5 | |
| AddSentBackbone=OS-Atlas-Base-7B2025.05 | 74 | 56.5 | 72 | 56 | 99.3 | 82.5 | 74.1 | 69.2 | 86.4 | 59.5 | 74.6 | 61.1 | |
| AddSentModel=Qwen2-VL-7B2025.05 | 73.8 | 41.9 | 71.1 | 55.3 | 99.1 | 74.4 | 73.7 | 69.3 | 86 | 58.3 | 74.3 | 51.6 | |
| AddSentModel=Qwen2-VL-2B2025.05 | 73.5 | 56.6 | 71 | 56.6 | 99.5 | 82.1 | 73.6 | 69 | 86.4 | 59 | 74.2 | 61.1 | |
| SynAttackModel=Qwen2-VL-2B2025.05 | 72.1 | 55.2 | 70.4 | 55.7 | 98.8 | 78.6 | 71.9 | 67.8 | 85.1 | 58.6 | 72.9 | 59.7 | |
| SynAttackModel=Qwen2-VL-7B2025.05 | 71.7 | 41 | 69.2 | 52.8 | 99.3 | 82.7 | 72.6 | 68.8 | 85.6 | 62.5 | 73 | 51.5 | |
| SynAttackBackbone=OS-Atlas-Base-7B2025.05 | 71.6 | 55.5 | 69.5 | 53.5 | 98.6 | 79.3 | 71.2 | 67.3 | 83.7 | 61.2 | 72.5 | 59.8 |