# Mobile Agent Decision-making on AndroidWorld (Evaluation set 116 templates) Source: [Mobile Agent Decision-making on AndroidWorld (Evaluation set 116 templates)]() Collection index: [llms.txt]() Data updated: 2026-03-10T10:16:28.167Z Evaluation of Mobile Agent Decision-making performance using the specific 116 template evaluation set from the AndroidWorld dataset, measured by Average Success Rate (SR). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Mobile Agent Decision-making | | Dataset (including any stated split) | AndroidWorld (Evaluation set 116 templates) | | Metric | Average Success Rate (SR) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 16 of 16 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Average Success Rate (SR)](): Higher is better; unit: PERCENT. ## Recorded results | Method | Average Success Rate (SR) | Result date | Report paper | | --- | --- | --- | --- | | MobileUse-72B | 62.9 | Not recorded | 69afa5a62d603e6de3e43ce4 | | Agent S2 | 54.3 | Not recorded | 69afa5a62d603e6de3e43ce4 | | Ours-7B | 49.1 | Not recorded | 69afa5a62d603e6de3e43ce4 | | UI-TARS-72B-SFT | 46.6 | Not recorded | 69afa5a62d603e6de3e43ce4 | | Ours-7B w/o curriculum learning | 45.1 | Not recorded | 69afa5a62d603e6de3e43ce4 | | Aria-UI+GPT-4o | 44.8 | Not recorded | 69afa5a62d603e6de3e43ce4 | | MobileGUI-32B | 44.8 | Not recorded | 69afa5a62d603e6de3e43ce4 | | UGround+GPT-4o | 44 | Not recorded | 69afa5a62d603e6de3e43ce4 | | AppVLM-3B | 37.8 | Not recorded | 69afa5a62d603e6de3e43ce4 | | Qwen2.5-VL-72B | 35 | Not recorded | 69afa5a62d603e6de3e43ce4 | | GPT-4o | 34.5 | Not recorded | 69afa5a62d603e6de3e43ce4 | | Qwen2.5-VL-32B | 31.5 | Not recorded | 69afa5a62d603e6de3e43ce4 | | MobileGUI-7B | 30 | Not recorded | 69afa5a62d603e6de3e43ce4 | | Claude Computer Use | 27.9 | Not recorded | 69afa5a62d603e6de3e43ce4 | | AGUVIS-72B | 26.1 | Not recorded | 69afa5a62d603e6de3e43ce4 | | UI-TARS-7B-SFT | 23 | Not recorded | 69afa5a62d603e6de3e43ce4 | ### MobileUse-72B Average Success Rate (SR): 62.9. Result ID: 69afef716dbb7075b958e99a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"API-Based":"false","Open-Source":"true","Reproducible":"true","Model Scale Category":"32B/72B Models"}. This record does not assert independent reproduction. ### Agent S2 Average Success Rate (SR): 54.3. Result ID: 69afef716dbb7075b958e994. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"API-Based":"true","Open-Source":"false","Reproducible":"false","Model Scale Category":"Proprietary model"}. This record does not assert independent reproduction. ### Ours-7B Average Success Rate (SR): 49.1. Result ID: 69afef716dbb7075b958e99f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"API-Based":"false","Open-Source":"true","Reproducible":"true","Curriculum Learning":"true","Model Scale Category":"2B/7B Models"}. This record does not assert independent reproduction. ### UI-TARS-72B-SFT Average Success Rate (SR): 46.6. Result ID: 69afef716dbb7075b958e999. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"API-Based":"false","Open-Source":"true","Reproducible":"true","Model Scale Category":"32B/72B Models"}. This record does not assert independent reproduction. ### Ours-7B w/o curriculum learning Average Success Rate (SR): 45.1. Result ID: 69afef716dbb7075b958e99e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"API-Based":"false","Open-Source":"true","Reproducible":"true","Curriculum Learning":"false","Model Scale Category":"2B/7B Models"}. This record does not assert independent reproduction. ### Aria-UI+GPT-4o Average Success Rate (SR): 44.8. Result ID: 69afef716dbb7075b958e993. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"API-Based":"true","Open-Source":"true","Reproducible":"true","Model Scale Category":"Proprietary model"}. This record does not assert independent reproduction. ### MobileGUI-32B Average Success Rate (SR): 44.8. Result ID: 69afef716dbb7075b958e996. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"API-Based":"false","Open-Source":"false","Reproducible":"false","Model Scale Category":"32B/72B Models"}. This record does not assert independent reproduction. ### UGround+GPT-4o Average Success Rate (SR): 44. Result ID: 69afef716dbb7075b958e992. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"API-Based":"true","Open-Source":"true","Reproducible":"true","Model Scale Category":"Proprietary model"}. This record does not assert independent reproduction. ### AppVLM-3B Average Success Rate (SR): 37.8. Result ID: 69afef716dbb7075b958e99b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"API-Based":"false","Open-Source":"false","Reproducible":"false","Model Scale Category":"2B/7B Models"}. This record does not assert independent reproduction. ### Qwen2.5-VL-72B Average Success Rate (SR): 35. Result ID: 69afef716dbb7075b958e998. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"API-Based":"false","Open-Source":"true","Reproducible":"true","Model Scale Category":"32B/72B Models"}. This record does not assert independent reproduction. ### GPT-4o Average Success Rate (SR): 34.5. Result ID: 69afef716dbb7075b958e990. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"API-Based":"true","Open-Source":"false","Reproducible":"true","Model Scale Category":"Proprietary model"}. This record does not assert independent reproduction. ### Qwen2.5-VL-32B Average Success Rate (SR): 31.5. Result ID: 69afef716dbb7075b958e995. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"API-Based":"false","Open-Source":"true","Reproducible":"true","Model Scale Category":"32B/72B Models"}. This record does not assert independent reproduction. ### MobileGUI-7B Average Success Rate (SR): 30. Result ID: 69afef716dbb7075b958e99c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"API-Based":"false","Open-Source":"false","Reproducible":"false","Model Scale Category":"2B/7B Models"}. This record does not assert independent reproduction. ### Claude Computer Use Average Success Rate (SR): 27.9. Result ID: 69afef716dbb7075b958e991. [Reporting paper]() Recorded evaluation configuration: {"API-Based":"true","Open-Source":"false","Reproducible":"true","Model Scale Category":"Proprietary model"}. This record does not assert independent reproduction. ### AGUVIS-72B Average Success Rate (SR): 26.1. Result ID: 69afef716dbb7075b958e997. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"API-Based":"false","Open-Source":"false","Reproducible":"false","Model Scale Category":"32B/72B Models"}. This record does not assert independent reproduction. ### UI-TARS-7B-SFT Average Success Rate (SR): 23. Result ID: 69afef716dbb7075b958e99d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"API-Based":"false","Open-Source":"true","Reproducible":"true","Model Scale Category":"2B/7B Models"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.