Average Execution Time for Long-Horizon Composite Skills on OSExpert-Eval
18Execution Time (GIMP)Human Expert
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Human ExpertAgent Type=Human Agent2026.03 | 18 | 25 | |
| OSExpert (Explore w/ Qwen-3-VL-8B)Agent Type=Agentic Framework2026.03 | 32 | 29 | |
| OSExpert (Exploration w/ GPT-5 & UI-TARS)Agent Type=Agentic Framework2026.03 | 34 | 27 | |
| OSExpert (Explore w/ Qwen-3-VL-8B, wo/ Fast Planner)Agent Type=Agentic Framework2026.03 | 46 | 31 | |
| OSExpert (Explore w/ Qwen-3-VL-8B, wo/ Boundary Check)Agent Type=Agentic Framework2026.03 | 84 | 91 | |
| Qwen-3-VL-8BAgent Type=General Model2026.03 | 139 | 136 | |
| OpenCUA-7BAgent Type=Specialized Model2026.03 | 169 | 193 | |
| Computer-Use-PreviewAgent Type=Specialized Model2026.03 | 252 | 240 | |
| Agent-S3 w/ GPT-5Agent Type=Agentic Framework2026.03 | 998 | 1,231 | |
| CoAct-1Agent Type=Agentic Framework2026.03 | 1,260 | 1,104 |