GUI Automation on OSWorld Verified (test)
61.92Overall Success RateSeed-1.8
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Seed-1.8Steps=1002026.02 | 61.92 | 63 | 53.8 | 72.3 | 68 | 82.5 | 49 | 70.8 | 60 | 58.2 | 73.9 | |
| Claude-Sonnet-4.5Steps=502026.02 | 58.1 | 56.4 | 57.7 | 66 | 57.5 | 65.2 | 47 | 70.8 | 66.7 | 52.9 | 69.6 | |
| UltraCUA-32B-RLModel Category=Agentic Model, Open-Source=true, Max Steps=502025.10 | 43.7 | — | — | — | — | — | — | — | — | — | — | |
| UltraCUA-32B-SFTModel Category=Agentic Model, Open-Source=true, Max Steps=502025.10 | 41.5 | — | — | — | — | — | — | — | — | — | — | |
| UltraCUA-32B-RLModel Category=Agentic Model, Open-Source=true, Max Steps=152025.10 | 41 | — | — | — | — | — | — | — | — | — | — | |
| UltraCUA-32B-SFTModel Category=Agentic Model, Open-Source=true, Max Steps=152025.10 | 39 | — | — | — | — | — | — | — | — | — | — | |
| TreeCUA-DPO-7BSteps=502026.02 | 36.6 | 39.1 | 76.9 | 25.5 | 29.8 | 47.8 | 15.1 | 54.2 | 53.3 | 47.1 | 60.9 | |
| Claude 3.7 SonnetModel Category=General Model, Open-Source=false, Max Steps=502025.10 | 35.8 | — | — | — | — | — | — | — | — | — | — | |
| OpenCUA-32B†Model Category=Agentic Model, Open-Source=true, Max Steps=502025.10 | 35.1 | — | — | — | — | — | — | — | — | — | — | |
| TreeCUA-7BSteps=502026.02 | 34.6 | 28.3 | 76.9 | 27.7 | 40.4 | 43.5 | 14 | 58.3 | 33.3 | 41.2 | 47.8 | |
| Agent S2Model Category=Multi-Agent Framework, Open-Source=true, Max Steps=502025.10 | 34.5 | — | — | — | — | — | — | — | — | — | — | |
| OpenCUA-32BModel Category=Agentic Model, Open-Source=true, Max Steps=502025.10 | 34.1 | — | — | — | — | — | — | — | — | — | — | |
| OpenCUA-32B†Model Category=Agentic Model, Open-Source=true, Max Steps=152025.10 | 33.3 | — | — | — | — | — | — | — | — | — | — | |
| OpenAI CUAModel Category=Agentic Model, Open-Source=false, Max Steps=502025.10 | 31.3 | — | — | — | — | — | — | — | — | — | — | |
| UltraCUA-7B-RLModel Category=Agentic Model, Open-Source=true, Max Steps=502025.10 | 30.2 | — | — | — | — | — | — | — | — | — | — | |
| OpenCUA-32BModel Category=Agentic Model, Open-Source=true, Max Steps=152025.10 | 29.7 | — | — | — | — | — | — | — | — | — | — | |
| UltraCUA-7BSteps=152026.02 | 28.9 | 41.2 | 50 | 13.9 | 27.1 | 55.4 | 10.6 | 37 | 33.6 | 43.3 | 46.7 | |
| UltraCUA-7B-RLModel Category=Agentic Model, Open-Source=true, Max Steps=152025.10 | 28.9 | — | — | — | — | — | — | — | — | — | — | |
| UltraCUA-7B-SFTModel Category=Agentic Model, Open-Source=true, Max Steps=502025.10 | 28.5 | — | — | — | — | — | — | — | — | — | — | |
| OpenCUA-7BModel Category=Agentic Model, Open-Source=true, Max Steps=502025.10 | 28.2 | — | — | — | — | — | — | — | — | — | — | |
| UI-TARS-1.5-7BModel Category=Agentic Model, Open-Source=true, Max Steps=502025.10 | 27.3 | — | — | — | — | — | — | — | — | — | — | |
| Claude 3.7 SonnetModel Category=General Model, Open-Source=false, Max Steps=152025.10 | 27.1 | — | — | — | — | — | — | — | — | — | — | |
| Jedi-7B w/ GPT-4oModel Category=Multi-Agent Framework, Open-Source=true, Max Steps=502025.10 | 27 | — | — | — | — | — | — | — | — | — | — | |
| Agent S2Model Category=Multi-Agent Framework, Open-Source=true, Max Steps=152025.10 | 27 | — | — | — | — | — | — | — | — | — | — | |
| UltraCUA-7B-SFTModel Category=Agentic Model, Open-Source=true, Max Steps=152025.10 | 27 | — | — | — | — | — | — | — | — | — | — | |
| Jedi-7B w/ GPT-4oModel Category=Multi-Agent Framework, Open-Source=true, Max Steps=152025.10 | 26.8 | — | — | — | — | — | — | — | — | — | — | |
| UI-TARS-1.5-7B†Model Category=Agentic Model, Open-Source=true, Max Steps=502025.10 | 26.1 | — | — | — | — | — | — | — | — | — | — | |
| OpenAI CUAModel Category=Agentic Model, Open-Source=false, Max Steps=152025.10 | 26 | — | — | — | — | — | — | — | — | — | — | |
| UI-TARS-72B-DPOModel Category=Agentic Model, Open-Source=true, Max Steps=502025.10 | 25.8 | — | — | — | — | — | — | — | — | — | — | |
| UI-TARS-1.5-7BSteps=502026.02 | 25.1 | 28.8 | 50 | 4.3 | 36.1 | 39.1 | 9.8 | 25 | 46.7 | 18.8 | 47.8 | |
| UI-TARS-1.5-7BModel Category=Agentic Model, Open-Source=true, Max Steps=152025.10 | 24.5 | — | — | — | — | — | — | — | — | — | — | |
| OpenCUA-7BSteps=152026.02 | 24.3 | 36.9 | 50 | 10.6 | 36.1 | 26.1 | 6.5 | 29.2 | 53.3 | 29.4 | 43.5 | |
| OpenCUA-7BModel Category=Agentic Model, Open-Source=true, Max Steps=152025.10 | 24.3 | — | — | — | — | — | — | — | — | — | — | |
| UI-TARS-72B-DPOModel Category=Agentic Model, Open-Source=true, Max Steps=152025.10 | 24 | — | — | — | — | — | — | — | — | — | — | |
| UI-TARS-1.5-7B†Model Category=Agentic Model, Open-Source=true, Max Steps=152025.10 | 23.4 | — | — | — | — | — | — | — | — | — | — | |
| o3Model Category=General Model, Open-Source=false, Max Steps=502025.10 | 17.2 | — | — | — | — | — | — | — | — | — | — | |
| ScaleCUA-7BSteps=502026.02 | 15 | — | — | — | — | — | — | — | — | — | — | |
| o3Model Category=General Model, Open-Source=false, Max Steps=152025.10 | 9.1 | — | — | — | — | — | — | — | — | — | — | |
| Qwen2.5-VL-7BSteps=502026.02 | 5.5 | 8.7 | 11.5 | 0 | 0 | 4.3 | 1.1 | 8.3 | 6.7 | 17.6 | 21.7 | |
| Qwen2.5-VL-72BModel Category=General Model, Open-Source=true, Max Steps=152025.10 | 4.4 | — | — | — | — | — | — | — | — | — | — |