Fine-Grained Action Execution on OSExpert-Eval (Execution Time)
22GIMP Execution Time (s)Human Expert
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Human ExpertAgent Type=Human Agent2026.03 | 22 | 11 | |
| OSExpert (Explore w/ Qwen-3-VL-8B)Agent Type=Agentic Framework2026.03 | 35 | 39 | |
| OSExpert (Exploration w/ GPT-5 & UI-TARS)Agent Type=Agentic Framework2026.03 | 35 | 37 | |
| OSExpert (Explore w/ Qwen-3-VL-8B, wo/ Fast Planner)Agent Type=Agentic Framework2026.03 | 38 | 44 | |
| OSExpert (Explore w/ Qwen-3-VL-8B, wo/ Boundary Check)Agent Type=Agentic Framework2026.03 | 61 | 82 | |
| Qwen-3-VL-8BAgent Type=General Model2026.03 | 83 | 138 | |
| OpenCUA-7BAgent Type=Specialized Model2026.03 | 162 | 173 | |
| Computer-Use-PreviewAgent Type=Specialized Model2026.03 | 256 | 282 | |
| Agent-S3 w/ GPT-5Agent Type=Agentic Framework2026.03 | 702 | 633 | |
| CoAct-1Agent Type=Agentic Framework2026.03 | 911 | 782 |