Agentic Phone Use on 150-task Real-phone Human Evaluation Cross-App
48Task Success RateGemini 3.1 Pro
Evaluation results
| Method | Links | |
|---|---|---|
| Gemini 3.1 Pro2026.06 | 48 | |
| GPT-5.42026.06 | 32 | |
| Seed 2.0 Pro2026.06 | 30 | |
| PhoneBuddy-4B-SFTParameters=4B, Training=supervised fine-tuning baseline2026.06 | 22 | |
| PhoneBuddy-4B-RealParameters=4B, Training=real-app RL2026.06 | 20 | |
| PhoneBuddy-4B-Real+MockParameters=4B, Training=mixed RL in real-app and mock-app2026.06 | 18 | |
| Claude Opus 4.72026.06 | 16 |