Mobile GUI Task Execution on AITZ zero-shot
91.17Scroll RateOS-Kairos
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| OS-KairosAPI=false, setting=zero-shot2025.02 | 91.17 | 73.51 | 91.65 | 98.43 | 89.46 | 99.2 | 96.81 | 87.54 | 24.51 | |
| Auto-UIAPI=false, setting=zero-shot2025.02 | 74.88 | 49.09 | 60.12 | 44.37 | 12.72 | 73 | 73.79 | 34.46 | — | |
| CogAgentAPI=false, setting=zero-shot2025.02 | 56.41 | 48.3 | 4.76 | 79.9 | 51.5 | 67.4 | 65.86 | 44.52 | — | |
| OS-Atlas-Pro-7BAPI=false, setting=zero-shot2025.02 | 27.4 | 0.66 | 5.16 | 93.31 | 34.87 | 85.2 | 85.2 | 33.66 | 0 | |
| GPT-4oAPI=true, setting=zero-shot2025.02 | 24.17 | 23.84 | 0 | 63.8 | 27.71 | 35.2 | 58.32 | 22.69 | 0 | |
| Qwen2-VL-7BAPI=false, setting=zero-shot2025.02 | 18.64 | 21.19 | 0 | 71.05 | 32.89 | 82.8 | 66.28 | 28.25 | 0 | |
| GLM-4V-PlusAPI=true, setting=zero-shot2025.02 | 11.65 | 7.28 | 0 | 79.15 | 27.65 | 43.8 | 68.95 | 20.92 | 0 | |
| Qwen-VL-MAXAPI=true, setting=zero-shot2025.02 | 7.89 | 13.04 | 10.2 | — | 72.3 | 34.04 | — | 52.41 | — |