Mobile Agent Decision-making on AndroidWorld (Evaluation set 116 templates)
62.9Average Success Rate (SR)MobileUse-72B
Evaluation results
| Method | Links | |
|---|---|---|
| MobileUse-72BAPI-Based=false, Open-Source=true, Reproducible=true, Model Scale Category=32B/72B Models2026.03 | 62.9 | |
| Agent S2API-Based=true, Open-Source=false, Reproducible=false, Model Scale Category=Proprietary model2026.03 | 54.3 | |
| Ours-7BAPI-Based=false, Open-Source=true, Reproducible=true, Curriculum Learning=true, Model Scale Category=2B/7B Models2026.03 | 49.1 | |
| UI-TARS-72B-SFTAPI-Based=false, Open-Source=true, Reproducible=true, Model Scale Category=32B/72B Models2026.03 | 46.6 | |
| Ours-7B w/o curriculum learningAPI-Based=false, Open-Source=true, Reproducible=true, Curriculum Learning=false, Model Scale Category=2B/7B Models2026.03 | 45.1 | |
| Aria-UI+GPT-4oAPI-Based=true, Open-Source=true, Reproducible=true, Model Scale Category=Proprietary model2026.03 | 44.8 | |
| MobileGUI-32BAPI-Based=false, Open-Source=false, Reproducible=false, Model Scale Category=32B/72B Models2026.03 | 44.8 | |
| UGround+GPT-4oAPI-Based=true, Open-Source=true, Reproducible=true, Model Scale Category=Proprietary model2026.03 | 44 | |
| AppVLM-3BAPI-Based=false, Open-Source=false, Reproducible=false, Model Scale Category=2B/7B Models2026.03 | 37.8 | |
| Qwen2.5-VL-72BAPI-Based=false, Open-Source=true, Reproducible=true, Model Scale Category=32B/72B Models2026.03 | 35 | |
| GPT-4oAPI-Based=true, Open-Source=false, Reproducible=true, Model Scale Category=Proprietary model2026.03 | 34.5 | |
| Qwen2.5-VL-32BAPI-Based=false, Open-Source=true, Reproducible=true, Model Scale Category=32B/72B Models2026.03 | 31.5 | |
| MobileGUI-7BAPI-Based=false, Open-Source=false, Reproducible=false, Model Scale Category=2B/7B Models2026.03 | 30 | |
| Claude Computer UseAPI-Based=true, Open-Source=false, Reproducible=true, Model Scale Category=Proprietary model2026.03 | 27.9 | |
| AGUVIS-72BAPI-Based=false, Open-Source=false, Reproducible=false, Model Scale Category=32B/72B Models2026.03 | 26.1 | |
| UI-TARS-7B-SFTAPI-Based=false, Open-Source=true, Reproducible=true, Model Scale Category=2B/7B Models2026.03 | 23 |