Mobile-use agent task completion and intent alignment on OS-Kairos user-specific Modified
69.74Success Rate (SR)IFRAgent
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| IFRAgentBase Model=OS-Atlas-7B-Pro2025.08 | 69.74 | 85.31 | 68.42 | |
| UI-TARS-1.5-7BBase Model=UI-TARS-1.5-7B2025.08 | 61.49 | 75.86 | 53.24 | |
| IFRAgentBase Model=UI-TARS-1.5-7B2025.08 | 60.75 | 77.04 | 58.8 | |
| OS-Atlas-7B-ProBase Model=OS-Atlas-7B-Pro2025.08 | 58.85 | 79.52 | 53.78 | |
| IFRAgentBase Model=GPT-4o+OCR model2025.08 | 56.33 | 77.36 | 53.75 | |
| IFRAgentBase Model=Qwen2.5-VL-7B2025.08 | 51.64 | 59.62 | 50.31 | |
| GPT-4o+OCR modelBase Model=GPT-4o+OCR model2025.08 | 40.65 | 68.92 | 37.67 | |
| Qwen2.5-VL-7BBase Model=Qwen2.5-VL-7B2025.08 | 24.27 | 27.77 | 23.64 |