Action Prediction on AndroidControl High v2
68.59Pass@1 Step AccuracyMem-W-8B
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Mem-W-8BBackbone=UI-Venus-1.5-8B, Memory Augmentation=Mem-W2026.05 | 68.59 | 81.16 | 71.3 | 80.4 | 88.94 | 82.96 | |
| GUI-Libra-8BModel Category=Open-source2026.05 | 64.3 | — | — | 70.6 | — | — | |
| Mem-W-4BBackbone=Qwen3-VL-4B, Memory Augmentation=Mem-W2026.05 | 63.07 | 74.62 | 70.85 | 74.87 | 83.92 | 80.72 | |
| GPT-5 + UGroundModel Category=Proprietary, Grounding Module=UGround2026.05 | 61.3 | — | — | 69.4 | — | — | |
| UI-Venus-1.5-8BModel Category=Baseline2026.05 | 61.06 | 74.62 | 61.06 | 70.1 | 80.4 | 82.51 | |
| Step-GUI-4BModel Category=Open-source2026.05 | 59.3 | 74.37 | 69.96 | 71.36 | 83.17 | 80.72 | |
| GUI-Libra-7BModel Category=Open-source2026.05 | 59.3 | — | — | 67.3 | — | — | |
| Qwen3-VL-32BModel Category=Open-source2026.05 | 59.05 | 74.62 | 69.96 | 70.1 | 82.16 | 77.58 | |
| GPT-4.1 + UGroundModel Category=Proprietary, Grounding Module=UGround2026.05 | 57.5 | — | — | 63.3 | — | — | |
| GPT-4o + UGroundModel Category=Proprietary, Grounding Module=UGround2026.05 | 57 | — | — | 66.3 | — | — | |
| Qwen3-VL-8BModel Category=Open-source2026.05 | 54.8 | 69.35 | 67.71 | 66.1 | 77.89 | 79.82 | |
| GPT-5-mini + UGroundModel Category=Proprietary, Grounding Module=UGround2026.05 | 52.8 | — | — | 58.8 | — | — | |
| UI-Venus-1.5-30B-A3BModel Category=Open-source2026.05 | 51.26 | 67.84 | 64.13 | 64.07 | 74.62 | 75.34 | |
| Qwen3-VL-4BModel Category=Baseline2026.05 | 49.3 | 63.57 | 69.06 | 63.3 | 75.88 | 77.13 | |
| Qwen3-VL-2BModel Category=Open-source2026.05 | 45.48 | 64.07 | 59.64 | 59.8 | 73.62 | 73.99 | |
| UI-TARS-1.5-7BModel Category=Open-source2026.05 | 45.2 | 70.6 | 57.4 | 63.1 | 84.17 | 72.65 | |
| MAI-UI-2BModel Category=Open-source2026.05 | 42.71 | 61.06 | 66.37 | 43.47 | 65.33 | 62.78 | |
| GUI-Owl-1.5-8BModel Category=Open-source2026.05 | 40.45 | 59.05 | 60.09 | 52.76 | 69.6 | 73.09 | |
| GUI-R1-3BModel Category=Open-source2026.05 | 40 | — | — | 54 | — | — | |
| GUI-R1-7BModel Category=Open-source2026.05 | 39.7 | — | — | 56.3 | — | — | |
| Aguvis-7BModel Category=Open-source2026.05 | 37.7 | — | — | 43.7 | — | — | |
| GLM-4.1V-9B-ThinkingModel Category=Open-source2026.05 | 37.2 | 67.84 | 48.43 | 49 | 74.87 | 59.64 |