Mobile GUI Automation on AndroidWorld
76.9Overall Success RateAutoRPA
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| AutoRPAModel=Claude-sonnet-4.5, Observation Type=Screenshot+A11y tree2026.05 | 76.9 | — | — | — | — | — | 36.1 | |
| ReAct†Model=Claude-sonnet-4.5, Observation Type=Screenshot+A11y tree2026.05 | 76.7 | — | — | — | — | — | 146 | |
| AutoRPAModel=GPT-52026.05 | 75.9 | — | — | — | — | 4.35 | 30.6 | |
| AutoRPAModel=GPT-5, Observation Type=Screenshot+A11y tree2026.05 | 75.9 | — | — | — | — | — | 30.6 | |
| ReAct†Model=GPT-52026.05 | 74.1 | — | — | — | — | 8.57 | 142.5 | |
| ReAct†Model=GPT-5, Observation Type=Screenshot+A11y tree2026.05 | 74.1 | — | — | — | — | — | 142.5 | |
| Mobile-Agent-v3Model=GUI-Owl-32B, Observation Type=Screenshot2026.05 | 73.3 | — | — | — | — | — | — | |
| AutoRPA (code only)Model=GPT-52026.05 | 70.7 | — | — | — | — | 2.72 | 6.2 | |
| AutoRPA (code only)Model=GPT-5, Observation Type=Screenshot+A11y tree2026.05 | 70.7 | — | — | — | — | — | 6.2 | |
| AutoRPAModel=Claude-sonnet-4.5, Observation Type=Screenshot2026.05 | 70.1 | — | — | — | — | — | 33.5 | |
| Gemini-2.5-CUModel=Gemini 2.5 Computer Use, Observation Type=Screenshot2026.05 | 69.7 | — | — | — | — | — | — | |
| ReAct†Model=Claude-sonnet-4.5, Observation Type=Screenshot2026.05 | 69 | — | — | — | — | — | 145.1 | |
| Agent S3Model=GPT-5, Observation Type=Screenshot2026.05 | 68.1 | — | — | — | — | — | — | |
| M3AModel=GPT-52026.05 | 57 | — | — | — | — | 5.56 | 164.6 | |
| M3AModel=GPT-5, Observation Type=Screenshot+A11y tree2026.05 | 57 | — | — | — | — | — | 164.6 | |
| CRAFT-GUI-32BTraining Stage=stage32025.08 | 51.7 | — | — | — | — | — | — | |
| AutoRPAModel=GPT-4.12026.05 | 51.7 | — | — | — | — | 1.81 | 12.8 | |
| ReAct†Model=GPT-4.12026.05 | 50 | — | — | — | — | 3.91 | 68.7 | |
| M3AModel=GPT-4.12026.05 | 48.3 | — | — | — | — | 2.23 | 103.4 | |
| AutoRPA (code only)Model=GPT-4.12026.05 | 47.2 | — | — | — | — | 1.42 | 2.7 | |
| UI-TARS-72B2025.08 | 46.6 | — | — | — | — | — | — | |
| Zero-ShotBase Model=GPT-4o [20]2026.03 | 45.22 | 42.86 | 66.67 | 77.27 | 29.09 | — | — | |
| CRAFT-GUI-32BTraining Stage=stage22025.08 | 44.8 | — | — | — | — | — | — | |
| CRAFT-GUI-32BTraining Stage=stage12025.08 | 41.4 | — | — | — | — | — | — | |
| SeeActModel=GPT-52026.05 | 40.3 | — | — | — | — | 12.43 | 134.5 | |
| SeeActModel=GPT-5, Observation Type=Screenshot+A11y tree2026.05 | 40.3 | — | — | — | — | — | 134.5 | |
| UI-Copilot-7Btool calling=true2026.04 | 39.1 | — | — | — | — | — | — | |
| AutoRPAModel=GPT-4o2026.05 | 37 | — | — | — | — | 1.76 | 14.7 | |
| ReAct†Model=GPT-4o2026.05 | 35.2 | — | — | — | — | 4.97 | 79.9 | |
| Qwen2.5-VL-72B2025.08 | 35 | — | — | — | — | — | — | |
| GPT-4o2025.08 | 34.5 | — | — | — | — | — | — | |
| GPT-4o (SoM)2026.04 | 34.5 | — | — | — | — | — | — | |
| M3AModel=GPT-4o2026.05 | 34.5 | — | — | — | — | 2.83 | 120.2 | |
| AutoRPA (code only)Model=GPT-4o2026.05 | 34.3 | — | — | — | — | 1.08 | 2.6 | |
| UI-S1-7B2026.04 | 34 | — | — | — | — | — | — | |
| UI-TARS-7B2026.04 | 33 | — | — | — | — | — | — | |
| UI-Copilot-7Btool calling=false2026.04 | 32.2 | — | — | — | — | — | — | |
| ClaudeVersion=Claude-computer-use2025.08 | 27.9 | — | — | — | — | — | — | |
| Claude-CU (SoM)2026.04 | 27.9 | — | — | — | — | — | — | |
| Aguvis-72B2025.08 | 26.1 | — | — | — | — | — | — | |
| SeeActModel=GPT-4.12026.05 | 25.4 | — | — | — | — | 5.14 | 58.8 | |
| HATSBase Model=InternVL2-8B [4]2026.03 | 24.35 | 16.09 | 25.93 | 53.03 | 13.58 | — | — | |
| HATSBase Model=Qwen2-VL-7B [61]2026.03 | 24.35 | 18.39 | 30.77 | 51.52 | 9.88 | — | — | |
| HATSBase Model=InternVL2-4B [4]2026.03 | 22.6 | 17.24 | 38.46 | 40.91 | 8.64 | — | — | |
| Qwen2.5-VL-32B2025.08 | 22 | — | — | — | — | — | — | |
| Qwen2.5VL-7B2026.04 | 22 | — | — | — | — | — | — | |
| OS-Genesis-7B2026.04 | 17.4 | — | — | — | — | — | — | |
| AgentCPM-GUI-8B2026.04 | 16.4 | — | — | — | — | — | — | |
| UI-Copilot-3B2026.04 | 15.7 | — | — | — | — | — | — | |
| SeeActModel=GPT-4o2026.05 | 15.3 | — | — | — | — | 6.38 | 68.4 | |
| UI-R1-7B2026.04 | 15.1 | — | — | — | — | — | — | |
| OS-GenesisBase Model=InternVL2-8B [4]2026.03 | 14.81 | 13.79 | 0 | 47.62 | 4.08 | — | — | |
| OS-Atlas-7B2026.04 | 12.1 | — | — | — | — | — | — | |
| OS-GenesisBase Model=Qwen2-VL-7B [61]2026.03 | 11.5 | 17.86 | 0 | 31.82 | 1.85 | — | — | |
| OS-GenesisBase Model=InternVL2-4B [4]2026.03 | 11.3 | 7.14 | 11.11 | 36.36 | 3.64 | — | — | |
| Task-DrivenBase Model=Qwen2-VL-7B [61]2026.03 | 10.43 | 8.33 | 3.7 | 30.3 | 2.47 | — | — | |
| UI-R1-3B2026.04 | 8.2 | — | — | — | — | — | — | |
| Task-DrivenBase Model=InternVL2-4B [4]2026.03 | 7.9 | 3.49 | 0 | 19.7 | 3.7 | — | — | |
| ShowUI-2B2026.04 | 7 | — | — | — | — | — | — | |
| Task-Driven w. Self-InstructBase Model=InternVL2-8B [4]2026.03 | 6.25 | 5.95 | 0 | 13.64 | 3.09 | — | — | |
| Task-Driven w. Self-InstructBase Model=InternVL2-4B [4]2026.03 | 6.09 | 2.3 | 3.7 | 21.21 | 0.62 | — | — | |
| Task-Driven w. Self-InstructBase Model=Qwen2-VL-7B [61]2026.03 | 6.09 | 6.9 | 7.41 | 25.76 | 0.62 | — | — | |
| Qwen2.5VL-3B2026.04 | 5 | — | — | — | — | — | — | |
| Task-DrivenBase Model=InternVL2-8B [4]2026.03 | 3.48 | 1.15 | 0 | 12.12 | 0.62 | — | — | |
| Zero-ShotBase Model=Qwen2-VL-7B [61]2026.03 | 1.75 | 0 | 0 | 7.58 | 0 | — | — | |
| Zero-ShotBase Model=InternVL2-8B [4]2026.03 | 1.04 | 0 | 0 | 3.13 | 0 | — | — | |
| Zero-ShotBase Model=InternVL2-4B [4]2026.03 | 1.02 | 0 | 0 | 2.38 | 0 | — | — | |
| Qwen2VL-2B2026.04 | 0 | — | — | — | — | — | — |