Mobile device navigation on Popular Applications 1.0 (test)
65Success Rate (SR)ANDROIDGEN
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ANDROIDGENBase Model=GPT-4o, Fine-tuned=false2025.04 | 65 | 7.6 | |
| AppAgentBase Model=GPT-4o, Fine-tuned=false2025.04 | 57.5 | 6.7 | |
| ANDROIDGENBase Model=Llama-3-70B, Fine-tuned=true2025.04 | 52.5 | 7.4 | |
| M3ABase Model=GPT-4o, Fine-tuned=false2025.04 | 40 | 7.3 | |
| ANDROIDGENBase Model=GLM-4-9B, Fine-tuned=true2025.04 | 35 | 7.2 | |
| SeeActBase Model=GPT-4o, Fine-tuned=false2025.04 | 22.5 | 7.9 |