Unseen UI Generalization on OSExpert-Eval (Execution Time)
16Execution Time (Tableau, s)Human Expert
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Human ExpertAgent Type=Human Agent2026.03 | 16 | 48 | |
| OSExpert (Explore w/ Qwen-3-VL-8B)Agent Type=Agentic Framework2026.03 | 28 | 41 | |
| OSExpert (Exploration w/ GPT-5 & UI-TARS)Agent Type=Agentic Framework2026.03 | 28 | 44 | |
| OSExpert (Explore w/ Qwen-3-VL-8B, wo/ Fast Planner)Agent Type=Agentic Framework2026.03 | 34 | 49 | |
| OSExpert (Explore w/ Qwen-3-VL-8B, wo/ Boundary Check)Agent Type=Agentic Framework2026.03 | 75 | 84 | |
| Qwen-3-VL-8BAgent Type=General Model2026.03 | 135 | 153 | |
| OpenCUA-7BAgent Type=Specialized Model2026.03 | 216 | 235 | |
| Computer-Use-PreviewAgent Type=Specialized Model2026.03 | 361 | 296 | |
| Agent-S3 w/ GPT-5Agent Type=Agentic Framework2026.03 | 623 | 628 | |
| CoAct-1Agent Type=Agentic Framework2026.03 | 924 | 857 |