High-level instruction following on AndroidControl
72.7Step AccuracyFalcon-UI-7B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Falcon-UI-7BMode=PF2025.05 | 72.7 | — | |
| MobileIPLMode=Ours2025.05 | 72.7 | 77 | |
| UI-Tars-7BMode=PF2025.05 | 72.5 | 80.5 | |
| OS-Atlas-7BMode=PF2025.05 | 71.2 | 78.5 | |
| MobileIPL-7BMode=IPL2025.05 | 70 | — | |
| Qwen2-VL-7B(GRPO)Mode=RL2025.05 | 69.8 | 70.7 | |
| Qwen2-VL-7B (SFT)Mode=FT2025.05 | 69.1 | 68.5 | |
| Qwen2-VL-7B(GRPO)Mode=RL2025.05 | 68.1 | — | |
| InternVL-2-4BMode=FT2025.05 | 66.7 | 72.7 | |
| Qwen2-VL-7B(SFT)Mode=FT2025.05 | 61.4 | — | |
| OS-Atlas-7BMode=PF2025.05 | 60.7 | — | |
| PaLM 2S(full)Mode=FT2025.05 | 58.7 | — | |
| PaLM 2S(LoRA)Mode=FT2025.05 | 58.5 | — | |
| SphAgent-7BMode=PF2025.05 | 57.1 | — | |
| FedMobileAgentMode=PF2025.05 | 52.3 | — | |
| Aria-UI-7BMode=FT2025.05 | 10.2 | 43.2 |