# Mobile GUI Agents on AndroidWorld 138 tasks (test) Source: [Mobile GUI Agents on AndroidWorld 138 tasks (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T06:05:11.562Z Evaluation of Mobile GUI Agent performance on the specific 138-task subset of the AndroidWorld dataset, measured by Success Rate. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Mobile GUI Agents | | Dataset (including any stated split) | AndroidWorld 138 tasks (test) | | Metric | Success Rate | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Success Rate](): Higher is better; unit: PERCENT. ## Recorded results | Method | Success Rate | Result date | Report paper | | --- | --- | --- | --- | | UI-Mem-8B | 71.1 | Not recorded | 698575d4c97a0b3e3b5d5e5d | | Seed1.8 | 70.7 | Not recorded | 698575d4c97a0b3e3b5d5e5d | | Gemini-2.5-Pro | 69.7 | Not recorded | 698575d4c97a0b3e3b5d5e5d | | Step-GUI-8B | 67.7 | Not recorded | 698575d4c97a0b3e3b5d5e5d | | UI-Mem-8B | 66.8 | Not recorded | 698575d4c97a0b3e3b5d5e5d | | GUI-Owl-7B | 66.4 | Not recorded | 698575d4c97a0b3e3b5d5e5d | | UI-Tars-1.5 | 64.2 | Not recorded | 698575d4c97a0b3e3b5d5e5d | | UI-Mem-4B | 62.5 | Not recorded | 698575d4c97a0b3e3b5d5e5d | | Seed1.5-VL | 62.1 | Not recorded | 698575d4c97a0b3e3b5d5e5d | | UI-Mem-4B | 58.2 | Not recorded | 698575d4c97a0b3e3b5d5e5d | | MAI-UI-2B | 49.1 | Not recorded | 698575d4c97a0b3e3b5d5e5d | | UI-Venus-7B | 49.1 | Not recorded | 698575d4c97a0b3e3b5d5e5d | | Qwen3-VL-8B | 47.6 | Not recorded | 698575d4c97a0b3e3b5d5e5d | | Qwen3-VL-4B | 45.3 | Not recorded | 698575d4c97a0b3e3b5d5e5d | | Qwen3-VL-2B | 36.4 | Not recorded | 698575d4c97a0b3e3b5d5e5d | | UI-Tars-1.5-7B | 30 | Not recorded | 698575d4c97a0b3e3b5d5e5d | | Ferret-UI Lite-3B | 28 | Not recorded | 698575d4c97a0b3e3b5d5e5d | | ScaleCUA-3B | 23.7 | Not recorded | 698575d4c97a0b3e3b5d5e5d | ### UI-Mem-8B Success Rate: 71.1. Result ID: 699739e85698665ec48a0f71. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params.":"8B","Inference-time memory retrieval":true}. This record does not assert independent reproduction. ### Seed1.8 Success Rate: 70.7. Result ID: 699739e85698665ec48a0f63. [Reporting paper]() Recorded evaluation configuration: {"Params.":"-","Inference-time memory retrieval":false}. This record does not assert independent reproduction. ### Gemini-2.5-Pro Success Rate: 69.7. Result ID: 699739e85698665ec48a0f62. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params.":"-","Inference-time memory retrieval":false}. This record does not assert independent reproduction. ### Step-GUI-8B Success Rate: 67.7. Result ID: 699739e85698665ec48a0f6d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Params.":"8B","Inference-time memory retrieval":false}. This record does not assert independent reproduction. ### UI-Mem-8B Success Rate: 66.8. Result ID: 699739e85698665ec48a0f70. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params.":"8B","Inference-time memory retrieval":false}. This record does not assert independent reproduction. ### GUI-Owl-7B Success Rate: 66.4. Result ID: 699739e85698665ec48a0f6c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params.":"7B","Inference-time memory retrieval":false}. This record does not assert independent reproduction. ### UI-Tars-1.5 Success Rate: 64.2. Result ID: 699739e85698665ec48a0f61. [Reporting paper]() Recorded evaluation configuration: {"Params.":"-","Inference-time memory retrieval":false}. This record does not assert independent reproduction. ### UI-Mem-4B Success Rate: 62.5. Result ID: 699739e85698665ec48a0f6a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params.":"4B","Inference-time memory retrieval":true}. This record does not assert independent reproduction. ### Seed1.5-VL Success Rate: 62.1. Result ID: 699739e85698665ec48a0f60. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params.":"-","Inference-time memory retrieval":false}. This record does not assert independent reproduction. ### UI-Mem-4B Success Rate: 58.2. Result ID: 699739e85698665ec48a0f69. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params.":"4B","Inference-time memory retrieval":false}. This record does not assert independent reproduction. ### MAI-UI-2B Success Rate: 49.1. Result ID: 699739e85698665ec48a0f65. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params.":"2B","Inference-time memory retrieval":false}. This record does not assert independent reproduction. ### UI-Venus-7B Success Rate: 49.1. Result ID: 699739e85698665ec48a0f6b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params.":"7B","Inference-time memory retrieval":false}. This record does not assert independent reproduction. ### Qwen3-VL-8B Success Rate: 47.6. Result ID: 699739e85698665ec48a0f6f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params.":"8B","Inference-time memory retrieval":false}. This record does not assert independent reproduction. ### Qwen3-VL-4B Success Rate: 45.3. Result ID: 699739e85698665ec48a0f68. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params.":"4B","Inference-time memory retrieval":false}. This record does not assert independent reproduction. ### Qwen3-VL-2B Success Rate: 36.4. Result ID: 699739e85698665ec48a0f64. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params.":"2B","Inference-time memory retrieval":false}. This record does not assert independent reproduction. ### UI-Tars-1.5-7B Success Rate: 30. Result ID: 699739e85698665ec48a0f6e. [Reporting paper]() Recorded evaluation configuration: {"Params.":"7B","Inference-time memory retrieval":false}. This record does not assert independent reproduction. ### Ferret-UI Lite-3B Success Rate: 28. Result ID: 699739e85698665ec48a0f67. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params.":"3B","Inference-time memory retrieval":false}. This record does not assert independent reproduction. ### ScaleCUA-3B Success Rate: 23.7. Result ID: 699739e85698665ec48a0f66. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params.":"3B","Inference-time memory retrieval":false}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.