Mobile GUI Agent Task on AndroidLab
44.9Success RateUI-Mem-8B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| UI-Mem-8BParams=8B, inference-time memory retrieval=true2026.02 | 44.9 | 56 | 89.8 | 94.9 | |
| UI-Mem-8BParams=8B, inference-time memory retrieval=false2026.02 | 43.5 | 52.7 | 93.9 | 90.9 | |
| MobileRLParams=7B2026.02 | 42.5 | — | — | — | |
| UI-TARS-1.5-7BParams=7B2026.02 | 40.6 | 49.4 | 84.2 | 92.5 | |
| UI-Mem-4BParams=4B, inference-time memory retrieval=true2026.02 | 39.9 | 51.9 | 89.2 | 94.6 | |
| UI-Genie-AgentParams=7B2026.02 | 38.7 | 46.3 | 92.8 | 91.4 | |
| UI-Mem-4BParams=4B, inference-time memory retrieval=false2026.02 | 37.7 | 49.5 | 93 | 93.5 | |
| Qwen3-VL-4BParams=4B2026.02 | 37 | 48.2 | 93 | 90.5 | |
| AutoGLMParams=-2026.02 | 36.2 | — | — | — | |
| Qwen3-VL-8BParams=8B2026.02 | 34.8 | 45.3 | 97.7 | 91.8 | |
| GPT-4oParams=-2026.02 | 31.2 | 35 | 87.3 | 85.4 | |
| Claude-3.5-SonnetParams=-2026.02 | 29 | 32.7 | 113.4 | 81.2 | |
| UI-Genie-AgentParams=3B2026.02 | 28.8 | 35.4 | 91.3 | 90.6 | |
| Gemini-1.5-ProParams=-2026.02 | 16.7 | 18.5 | 106 | 91.5 | |
| Qwen2.5-VLParams=7B2026.02 | 14.9 | 18.7 | 70.6 | 76.8 | |
| Claude-3-OpusParams=-2026.02 | 13 | 15.1 | 81.4 | 83.9 | |
| CogVLM2-ftParams=19B2026.02 | 11.6 | 16.1 | 57.4 | 85.6 | |
| Gemini-1.0Params=-2026.02 | 10.9 | 12.6 | 72.5 | 76.7 | |
| LLaMA3.2-11B-VisionParams=11B2026.02 | 10.1 | 13 | 61.7 | 87.9 |