Mobile GUI Automation on GUI-Odyssey
87Success Rate (SR)UI-TARS-7B*
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| UI-TARS-7B*Training Data=Open-source Models W/ Private Human-Annotated Datasets2026.02 | 87 | — | — | 90.1 | |
| M2-Miner-7BTraining Data=Open-source Models W/ Auto-Mined Datasets2026.02 | 79.3 | — | — | 90.5 | |
| OdysseyAgentTraining Data=Open-source Models W/ Public Datasets2026.02 | 78.2 | — | — | — | |
| M2-Miner-3BTraining Data=Open-source Models W/ Auto-Mined Datasets2026.02 | 77.1 | — | — | 88.2 | |
| SpiritSight-8BTraining Data=Open-source Models W/ Public Datasets2026.02 | 75.8 | — | — | — | |
| UI-Venus-Navi-7BMethod Category=Specialized Models, Tuning Strategy=Parameter-tuned, Memory-augmented=false2026.01 | 65.93 | — | 74.52 | — | |
| OS-Atlas-7BTraining Data=Open-source Models W/ Public Datasets2026.02 | 62 | — | — | 84.5 | |
| GUI-Owl-7BTraining Data=Open-source Models W/ Auto-Mined Datasets2026.02 | 60.7 | — | — | 83.4 | |
| Qwen2.5-VL-7BTraining Data=Open-source Models W/ Public Datasets2026.02 | 60.3 | — | — | 83.7 | |
| Atlas-Pro-7BMethod Category=Specialized Models, Tuning Strategy=Parameter-tuned, Memory-augmented=false2026.01 | 58.82 | — | 67 | — | |
| Qwen2.5-VL-3BTraining Data=Open-source Models W/ Public Datasets2026.02 | 57.8 | — | — | 81.2 | |
| UI-TARS 1.0* + ICM-r2Base Model=UI-TARS 1.0, Refinement=ICM-r22026.01 | 56.3 | — | 78.5 | — | |
| InfiGUI-R1-3BMethod Category=Specialized Models, Tuning Strategy=Parameter-tuned, Memory-augmented=false2026.01 | 55.51 | — | 64.98 | — | |
| UI-TARS 1.0* + ICMBase Model=UI-TARS 1.0, Refinement=ICM2026.01 | 55.3 | — | 76.1 | — | |
| InfiGUI-R1Training Data=Open-source Models W/ Public Datasets2026.02 | 55 | — | — | 74.5 | |
| SeeClickMethod Type=SFT, Size=9.6B2026.01 | 53.9 | — | 52.4 | — | |
| GUI-R1-7B + CESMethod=Multi-Agent2025.11 | 53.69 | 79.24 | 63.82 | — | |
| UI-S1-7BCategory=Open-src, Scale=7B/8B2026.04 | 52.8 | 70.5 | 62.8 | — | |
| VeriGUI-7BCategory=Ours, Scale=7B2026.04 | 52.3 | 72.7 | 63.2 | — | |
| UI-TARS-7BCategory=Open-src, Scale=7B/8B2026.04 | 52 | 72.6 | 62.1 | — | |
| SWIRLMethod=Multi-Agent2025.11 | 51.65 | 74.87 | 66.39 | — | |
| UI-TARS 1.5* + ICM-r2Base Model=UI-TARS 1.5, Refinement=ICM-r22026.01 | 50.2 | — | 53.5 | — | |
| MAGNETMethod Category=Agentic Frameworks, Tuning Strategy=Training-free / Inference-only, Backbone=Qwen2.5-VL-32B, Memory-augmented=true2026.01 | 50.16 | — | 51.91 | — | |
| MAGNETMethod Category=Agentic Frameworks, Tuning Strategy=Training-free / Inference-only, Backbone=Gemini-2.5-Pro, Memory-augmented=true2026.01 | 49.74 | — | 57.31 | — | |
| Agent-SMethod Category=Agentic Frameworks, Tuning Strategy=Training-free / Inference-only, Backbone=Qwen2.5-VL-32B, Memory-augmented=true2026.01 | 49.21 | — | 50.74 | — | |
| Qwen2.5-VL-7BCategory=Open-src, Scale=7B/8B2026.04 | 48.5 | 72.3 | 55.8 | — | |
| COATMethod Category=Agentic Frameworks, Tuning Strategy=Training-free / Inference-only, Backbone=Qwen2.5-VL-32B, Memory-augmented=false2026.01 | 48.13 | — | 49.38 | — | |
| Doubao + ICM-r2Base Model=Doubao, Refinement=ICM-r22026.01 | 47.9 | — | 68 | — | |
| UI-TARS 1.5* + ICMBase Model=UI-TARS 1.5, Refinement=ICM2026.01 | 47.8 | — | 52.9 | — | |
| AgentCPM-GUI-8BCategory=Open-src, Scale=7B/8B2026.04 | 47.8 | 70.3 | 50.2 | — | |
| GUI-R1-7BMethod Category=Specialized Models, Tuning Strategy=Parameter-tuned, Memory-augmented=false2026.01 | 47.7 | — | 59.2 | — | |
| Doubao + ICMBase Model=Doubao, Refinement=ICM2026.01 | 46.9 | — | 68.4 | — | |
| Gemini-3-flashCategory=Closed-source2026.04 | 46.9 | 83.2 | 48.8 | — | |
| Qwen 2.5 VL + ICM-r2Base Model=Qwen 2.5 VL, Refinement=ICM-r22026.01 | 44.8 | — | 78.3 | — | |
| DoubaoMethod Type=N/A, Size=N/A2026.01 | 43.8 | — | 67.3 | — | |
| Qwen 2.5 VL + ICMBase Model=Qwen 2.5 VL, Refinement=ICM2026.01 | 43.7 | — | 77.2 | — | |
| VeriGUI-3BCategory=Ours, Scale=3B2026.04 | 43.4 | 65.5 | 54.5 | — | |
| UI-TARS 1.0*Method Type=N/A, Size=7B2026.01 | 43.3 | — | 71 | — | |
| GUI-R1-7B + GPT-5Method=Multi-Agent2025.11 | 42.47 | 70.37 | 49.92 | — | |
| GUI-R1Method Type=RFT, Size=3B2026.01 | 41.3 | — | 41.5 | — | |
| OS-Atlas-7BCategory=Open-src, Scale=7B/8B2026.04 | 41.3 | 70.2 | 63.5 | — | |
| Qwen 2.5 VLMethod Type=N/A, Size=7B2026.01 | 40.2 | — | 77.2 | — | |
| Qwen2.5-VL-3BCategory=Open-src, Scale=3B2026.04 | 39.9 | 62 | 47.4 | — | |
| GUI-R1-7BTraining Data=Open-source Models W/ Public Datasets2026.02 | 38.8 | — | — | 65.5 | |
| GUI-R1-7BMethod=RL2025.11 | 38.79 | 65.49 | 43.64 | — | |
| Aria-UIMethod Type=SFT, Size=3.9B2026.01 | 36.5 | — | 86.8 | — | |
| GUI-Owl-7BMethod=RL2025.11 | 35.82 | 60.6 | 45.96 | — | |
| OS-Genesis-7BCategory=Open-src, Scale=7B/8B2026.04 | 34.5 | 62.3 | 59.9 | — | |
| Qwen2.5-VL-7BMethod=Zero Shot2025.11 | 34.37 | 55.6 | 37.78 | — | |
| UI-TARS 1.5*Method Type=N/A, Size=7B2026.01 | 32.9 | — | 44.6 | — | |
| UI-R1Method Type=RFT, Size=3B2026.01 | 32.5 | — | 34.5 | — | |
| UI-R1-3BMethod=RL2025.11 | 32.49 | 52.16 | 34.46 | — | |
| UI-R1-3BCategory=Open-src, Scale=3B2026.04 | 32 | 62.5 | 52.6 | — | |
| GPT-5.1Category=Closed-source2026.04 | 27.9 | 68.1 | 34 | — | |
| OS-Atlas-BaseMethod Type=ZS, Size=7B2026.01 | 27 | — | 39.7 | — | |
| OS-Atlas-7BMethod=SFT2025.11 | 26.96 | 60.42 | 39.74 | — | |
| GPT4o + ICM-r2Base Model=GPT4o, Refinement=ICM-r22026.01 | 14.5 | — | 11.4 | — | |
| Aguvis-7BTraining Data=Open-source Models W/ Public Datasets2026.02 | 13.5 | — | — | 26.7 | |
| GPT4o + ICMBase Model=GPT4o, Refinement=ICM2026.01 | 13.4 | — | 10 | — | |
| GPT4oMethod Type=N/A, Size=N/A2026.01 | 11.4 | — | 11.6 | — | |
| GPT-4oMethod=Zero Shot2025.11 | 5.36 | 37.5 | 14.17 | — | |
| OS-Genesis-7BTraining Data=Open-source Models W/ Auto-Mined Datasets2026.02 | 3.6 | — | — | 11.7 | |
| ClaudeTraining Data=Closed-source Models2026.02 | — | — | — | 60.9 |