# Mobile UI Reasoning on AMEX Source: [Mobile UI Reasoning on AMEX]() Collection index: [llms.txt]() Data updated: 2026-03-09T09:00:34.221Z Evaluation of model performance on Mobile UI Reasoning tasks using the AMEX dataset, reporting component-specific scores (e.g., Gmail, Booking) and an overall score. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Mobile UI Reasoning | | Dataset (including any stated split) | AMEX | | Metric | Gmail Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 17 of 17 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Gmail Accuracy](): Higher is better; unit: PERCENT. - [Booking Accuracy](): Higher is better; unit: PERCENT. - [Music Accuracy](): Higher is better; unit: PERCENT. - [SHEIN Accuracy](): Higher is better; unit: PERCENT. - [News Accuracy](): Higher is better; unit: PERCENT. - [CM Accuracy](): Higher is better; unit: PERCENT. - [ToDo Accuracy](): Higher is better; unit: PERCENT. - [Signal Accuracy](): Higher is better; unit: PERCENT. - [Yelp Accuracy](): Higher is better; unit: PERCENT. - [Overall Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Gmail Accuracy | Result date | Report paper | | --- | --- | --- | --- | | CoME | 76.2 | Not recorded | 69a519842d603e6de3e3c9b6 | | OS-Atlas | 74.4 | Not recorded | 69a519842d603e6de3e3c9b6 | | UGround | 70.9 | Not recorded | 69a519842d603e6de3e3c9b6 | | UITars | 67.7 | Not recorded | 69a519842d603e6de3e3c9b6 | | Qwen2VL-MoE | 65.1 | Not recorded | 69a519842d603e6de3e3c9b6 | | AriaUI | 63.1 | Not recorded | 69a519842d603e6de3e3c9b6 | | SphAgent | 61.7 | Not recorded | 69a519842d603e6de3e3c9b6 | | UITars | 59.4 | Not recorded | 69a519842d603e6de3e3c9b6 | | Qwen2VL | 57.6 | Not recorded | 69a519842d603e6de3e3c9b6 | | ShowUI | 52.2 | Not recorded | 69a519842d603e6de3e3c9b6 | | UI-S1 | 47.3 | Not recorded | 69a519842d603e6de3e3c9b6 | | DeepSeekVL2 | 43.1 | Not recorded | 69a519842d603e6de3e3c9b6 | | MolmoE | 38.7 | Not recorded | 69a519842d603e6de3e3c9b6 | | Qwen2VL | 37.4 | Not recorded | 69a519842d603e6de3e3c9b6 | | GUI-R1 | 37.1 | Not recorded | 69a519842d603e6de3e3c9b6 | | GUI-R1 | 36.8 | Not recorded | 69a519842d603e6de3e3c9b6 | | SeeClick | 28.2 | Not recorded | 69a519842d603e6de3e3c9b6 | ### CoME Gmail Accuracy: 76.2. Result ID: 69ae8c1b6dbb7075b956dd40. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"#Params":"5B"}. This record does not assert independent reproduction. ### OS-Atlas Gmail Accuracy: 74.4. Result ID: 69ae8c1b6dbb7075b956dd37. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"#Params":"7B"}. This record does not assert independent reproduction. ### UGround Gmail Accuracy: 70.9. Result ID: 69ae8c1b6dbb7075b956dd36. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"#Params":"7B"}. This record does not assert independent reproduction. ### UITars Gmail Accuracy: 67.7. Result ID: 69ae8c1b6dbb7075b956dd39. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"#Params":"7B"}. This record does not assert independent reproduction. ### Qwen2VL-MoE Gmail Accuracy: 65.1. Result ID: 69ae8c1b6dbb7075b956dd3d. [Reporting paper]() Recorded evaluation configuration: {"#Params":"3B","hybrid-capabilities reasoning finetuned":"true"}. This record does not assert independent reproduction. ### AriaUI Gmail Accuracy: 63.1. Result ID: 69ae8c1b6dbb7075b956dd3e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"#Params":"3.9B"}. This record does not assert independent reproduction. ### SphAgent Gmail Accuracy: 61.7. Result ID: 69ae8c1b6dbb7075b956dd38. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"#Params":"7B","reported\_from":"Chai et al., 2024"}. This record does not assert independent reproduction. ### UITars Gmail Accuracy: 59.4. Result ID: 69ae8c1b6dbb7075b956dd32. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"#Params":"2B"}. This record does not assert independent reproduction. ### Qwen2VL Gmail Accuracy: 57.6. Result ID: 69ae8c1b6dbb7075b956dd35. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"#Params":"7B","hybrid-capabilities reasoning finetuned":"true"}. This record does not assert independent reproduction. ### ShowUI Gmail Accuracy: 52.2. Result ID: 69ae8c1b6dbb7075b956dd31. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"#Params":"2B"}. This record does not assert independent reproduction. ### UI-S1 Gmail Accuracy: 47.3. Result ID: 69ae8c1b6dbb7075b956dd3b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"#Params":"7B"}. This record does not assert independent reproduction. ### DeepSeekVL2 Gmail Accuracy: 43.1. Result ID: 69ae8c1b6dbb7075b956dd3f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"#Params":"4.5B","hybrid-capabilities reasoning finetuned":"true"}. This record does not assert independent reproduction. ### MolmoE Gmail Accuracy: 38.7. Result ID: 69ae8c1b6dbb7075b956dd3c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"#Params":"1B","hybrid-capabilities reasoning finetuned":"true"}. This record does not assert independent reproduction. ### Qwen2VL Gmail Accuracy: 37.4. Result ID: 69ae8c1b6dbb7075b956dd30. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"#Params":"2B","hybrid-capabilities reasoning finetuned":"true"}. This record does not assert independent reproduction. ### GUI-R1 Gmail Accuracy: 37.1. Result ID: 69ae8c1b6dbb7075b956dd33. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"#Params":"3B"}. This record does not assert independent reproduction. ### GUI-R1 Gmail Accuracy: 36.8. Result ID: 69ae8c1b6dbb7075b956dd3a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"#Params":"3B"}. This record does not assert independent reproduction. ### SeeClick Gmail Accuracy: 28.2. Result ID: 69ae8c1b6dbb7075b956dd34. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"#Params":"7B","reported\_from":"Chai et al., 2024"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.