Mobile Agent Interaction on MobileIAR Chinese users
51.97SROS-Atlas-7B-Pro
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| OS-Atlas-7B-ProCategory=Open-source, IFRAgent=Enabled2025.08 | 51.97 | 81.64 | 47.82 | |
| UI-TARS-7B-SFTCategory=Open-source, IFRAgent=Enabled2025.08 | 51.04 | 75.86 | 48.7 | |
| UI-TARS-1.5-7BCategory=Open-source, IFRAgent=Enabled2025.08 | 49.8 | 77.01 | 46.75 | |
| UI-TARS-7B-DPOCategory=Open-source, IFRAgent=Enabled2025.08 | 46.45 | 73.73 | 43.46 | |
| UI-TARS-1.5-7BCategory=Open-source, IFRAgent=Disabled2025.08 | 46.22 | 76.42 | 39.18 | |
| UI-TARS-7B-DPOCategory=Open-source, IFRAgent=Disabled2025.08 | 45.07 | 70.74 | 37.19 | |
| UI-TARS-7B-SFTCategory=Open-source, IFRAgent=Disabled2025.08 | 44.86 | 75 | 36.86 | |
| GUI-owl-7BCategory=Open-source, IFRAgent=Enabled2025.08 | 44.22 | 73.26 | 38.49 | |
| GPT-4o + OCR modelCategory=Close-source, IFRAgent=Enabled2025.08 | 44.19 | 78.06 | 41.4 | |
| Qwen3-VL-8BCategory=Open-source, IFRAgent=Enabled2025.08 | 43.01 | 70.21 | 37.36 | |
| OS-Atlas-7B-ProCategory=Open-source, IFRAgent=Disabled2025.08 | 42.22 | 76.92 | 35.67 | |
| GUI-owl-7BCategory=Open-source, IFRAgent=Disabled2025.08 | 39.32 | 70.21 | 35.07 | |
| Qwen2.5-VL-7BCategory=Open-source, IFRAgent=Enabled2025.08 | 38.27 | 47.27 | 36.13 | |
| Qwen3-VL-8BCategory=Open-source, IFRAgent=Disabled2025.08 | 38.08 | 65.57 | 32.09 | |
| GPT-4o + OCR modelCategory=Close-source, IFRAgent=Disabled2025.08 | 37.13 | 77.42 | 31.18 | |
| Qwen-VL-max + OCR modelCategory=Close-source, IFRAgent=Enabled2025.08 | 24.04 | 84.42 | 21.34 | |
| Qwen-VL-max + OCR modelCategory=Close-source, IFRAgent=Disabled2025.08 | 22 | 81.55 | 19.04 | |
| Qwen2.5-VL-7BCategory=Open-source, IFRAgent=Disabled2025.08 | 19.29 | 23.52 | 18.13 | |
| GLM-4v + OCR modelCategory=Close-source, IFRAgent=Enabled2025.08 | 4.05 | 73.71 | 3.03 | |
| GLM-4v + OCR modelCategory=Close-source, IFRAgent=Disabled2025.08 | 3.11 | 72.97 | 2.47 |