GUI Agent Task Completion on OSWorld 1.0 (test)
91.67Success Rate (OS)Claude Sonnet 4.6
Evaluation results
| Method | Links | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Claude Sonnet 4.6Model Category=Generalized Model, Max inference steps=50, Number of trials=32026.05 | 91.67 | — | 69.23 | 74.47 | 70.21 | 86.83 | — | 66.67 | 81.41 | 72.73 | 76.65 | — | — | — | — | — | |
| Kimi K2.6Model Category=Generalized Model, Max inference steps=50, Number of trials=32026.05 | 79.17 | — | 73.08 | 80.85 | 82.19 | 73.91 | — | 80 | 75.71 | 91.3 | 79.53 | — | — | — | — | — | |
| EvoCUA-32BModel Category=CUA Model, Max inference steps=50, Number of trials=32026.05 | 75 | — | 76.29 | 51.06 | 52.98 | 65.22 | — | 60 | 64.65 | 65.22 | 63.8 | — | — | — | — | — | |
| EvoCUA-8B + LEARNWEAKModel Category=CUA Model, Specialization=LEARNWEAK (LoRA), Max inference steps=50, Number of trials=32026.05 | 66.67 | — | 82.05 | 41.13 | 50.35 | 55.07 | — | 73.33 | 56.86 | 72.46 | 62.24 | — | — | — | — | — | |
| OpenCUA-32BModel Category=CUA Model, Max inference steps=50, Number of trials=32026.05 | 61.11 | — | 74.36 | 35.46 | 48.21 | 56.52 | — | 57.78 | 37.25 | 72.73 | 55.43 | — | — | — | — | — | |
| EvoCUA-8BModel Category=CUA Model, Max inference steps=50, Number of trials=32026.05 | 60.83 | — | 66.15 | 28.07 | 37.66 | 50.43 | — | 65.33 | 45.71 | 51.3 | 50.69 | — | — | — | — | — | |
| OpenCUA-7B + LEARNWEAKModel Category=CUA Model, Specialization=LEARNWEAK (LoRA), Max inference steps=50, Number of trials=32026.05 | 59.42 | — | 57.69 | 19.15 | 36.88 | 40.58 | — | 66.67 | 47.06 | 62.32 | 48.72 | — | — | — | — | — | |
| OpenCUA-32Bmaximum steps=152026.01 | 52.2 | 40.5 | 55.1 | 13.5 | 30.7 | 39.1 | 9.7 | 44.4 | 25 | 52.8 | 29.7 | — | — | — | — | — | |
| Claude-3.7-Sonnetmaximum steps=152026.01 | 50 | 41.2 | 34.6 | 8.5 | 29.7 | 39.1 | 10.8 | 33.3 | 35.3 | 43.5 | 27.1 | — | — | — | — | — | |
| UI-TARS-1.5-7Bmaximum steps=15, training tasks=400 tasks from INFINITEWEB2026.01 | 48.6 | 35.5 | 69.2 | 10.6 | 29.8 | 37.7 | 9 | 35.5 | 33.3 | 62.3 | 29.7 | — | — | — | — | — | |
| Computer-use-previewmaximum steps=152026.01 | 45.8 | 36.9 | 34.6 | 10.6 | 25.4 | 30.4 | 10.8 | 46.7 | 29.4 | 47.8 | 26 | — | — | — | — | — | |
| UI-TARS-1.5-7Bmaximum steps=15, training tasks=600 tasks from INFINITEWEB2026.01 | 45.8 | 36.9 | 69.2 | 12.8 | 29.8 | 47.8 | 9.7 | 40 | 35.3 | 65.2 | 31.4 | — | — | — | — | — | |
| UI-TARS-1.5-7Bmaximum steps=15, training tasks=200 tasks from INFINITEWEB2026.01 | 41.7 | 34.8 | 61.5 | 10 | 27.7 | 34.8 | 8 | 40 | 25.5 | 55.1 | 27.3 | — | — | — | — | — | |
| Qwen3.5-27BModel Category=Generalized Model, Max inference steps=50, Number of trials=32026.05 | 41.67 | — | 39.74 | 22.7 | 43.97 | 52.17 | — | 66.67 | 44.12 | 47.83 | 44.86 | — | — | — | — | — | |
| OpenCUA-7BModel Category=CUA Model, Max inference steps=50, Number of trials=32026.05 | 40 | — | 48.46 | 11.91 | 31.49 | 30.43 | — | 54.67 | 32.94 | 51.3 | 37.65 | — | — | — | — | — | |
| Claude-4-Sonnetmaximum steps=152026.01 | 37.5 | 36.9 | 46.2 | 17 | 36.2 | 43.5 | 9.7 | 66.7 | 38.5 | 60.9 | 31.2 | — | — | — | — | — | |
| UI-TARS-1.5-7Bmaximum steps=152026.01 | 34.8 | 22.9 | 51.9 | 11.7 | 29.7 | 39.1 | 3.8 | 26.7 | 34.2 | 63 | 24.5 | — | — | — | — | — | |
| Doubao-1.5-thinking-0428maximum steps=152026.01 | 30.4 | 44.4 | 46.2 | 13 | 31.8 | 39.1 | 4.8 | 66.7 | 23.5 | 56.5 | 28.1 | — | — | — | — | — | |
| Claude Sonnet 4.6Model Category=General-Purpose Models2026.06 | 22 | — | — | — | — | — | — | — | — | — | — | 72.1 | 88 | 33 | 58 | 56 | |
| SGCD-30B-A3B#Params=30B-A3B, Backbone=Qwen3-VL-30B-A3B backbone, Model Category=Backbone-Matched Models2026.06 | 18 | — | — | — | — | — | — | — | — | — | — | 58.4 | 78 | 17 | 55 | 43 | |
| EvoCUA-8B#Params=8B, Backbone=Qwen3-VL-8B backbone, Model Category=Backbone-Matched Models2026.06 | 18 | — | — | — | — | — | — | — | — | — | — | 46.1 | 44 | 25 | 56 | 24 | |
| GUI-Owl-1.5-8B-Instruct#Params=8B, Backbone=Qwen3-VL-8B backbone, Model Category=Backbone-Matched Models, Evaluation Protocol=reproduced2026.06 | 18 | — | — | — | — | — | — | — | — | — | — | 52.4 | 59 | 19 | 65 | 28 | |
| Kimi K2.5#Params=1T, Model Category=General-Purpose Models2026.06 | 17 | — | — | — | — | — | — | — | — | — | — | 63.3 | 81 | 29 | 59 | 43 | |
| Seed1.8Model Category=General-Purpose Models2026.06 | 16 | — | — | — | — | — | — | — | — | — | — | 61.9 | 81 | 29 | 58 | 39 | |
| DeepMiner-Mano-72B#Params=72B, Model Category=GUI-Specialized Models2026.06 | 16 | — | — | — | — | — | — | — | — | — | — | 53.9 | 74 | 25 | 57 | 23 | |
| SGCD-8B#Params=8B, Backbone=Qwen3-VL-8B backbone, Model Category=Backbone-Matched Models2026.06 | 16 | — | — | — | — | — | — | — | — | — | — | 55.1 | 75 | 18 | 58 | 32 | |
| OpenCUA-32B#Params=32B, Model Category=GUI-Specialized Models2026.06 | 15 | — | — | — | — | — | — | — | — | — | — | 35.6 | 36 | 20 | 45 | 12 | |
| OpenCUA-72B-preview#Params=72B, Model Category=GUI-Specialized Models2026.06 | 14 | — | — | — | — | — | — | — | — | — | — | 45.9 | 55 | 26 | 52 | 19 | |
| DART-GUI-7B#Params=7B, Model Category=GUI-Specialized Models2026.06 | 13 | — | — | — | — | — | — | — | — | — | — | 40.5 | 39 | 26 | 52 | 17 | |
| SGCD-10B#Params=10B, Backbone=STEP3-VL-10B backbone, Model Category=Backbone-Matched Models2026.06 | 13 | — | — | — | — | — | — | — | — | — | — | 53.2 | 69 | 19 | 59 | 32 | |
| GUI-Owl-7B#Params=7B, Model Category=GUI-Specialized Models2026.06 | 12 | — | — | — | — | — | — | — | — | — | — | 32.1 | 29 | 19 | 47 | 9 | |
| DeepMiner-Mano-7B#Params=7B, Model Category=GUI-Specialized Models2026.06 | 12 | — | — | — | — | — | — | — | — | — | — | 40.1 | 46 | 18 | 53 | 16 | |
| STEP3-VL-10B#Params=10B, Backbone=STEP3-VL-10B backbone, Model Category=Backbone-Matched Models, Evaluation Protocol=reproduced2026.06 | 12 | — | — | — | — | — | — | — | — | — | — | 24.1 | 17 | 13 | 39 | 6 | |
| OpenCUA-7B#Params=7B, Model Category=GUI-Specialized Models2026.06 | 11 | — | — | — | — | — | — | — | — | — | — | 28.7 | 28 | 18 | 34 | 13 | |
| Qwen3-VL-30B-A3B-Thinking#Params=30B-A3B, Backbone=Qwen3-VL-30B-A3B backbone, Model Category=Backbone-Matched Models, Evaluation Protocol=reproduced2026.06 | 11 | — | — | — | — | — | — | — | — | — | — | 31.3 | 40 | 13 | 41 | 8 | |
| Qwen3-VL-8B-Thinking#Params=8B, Backbone=Qwen3-VL-8B backbone, Model Category=Backbone-Matched Models, Evaluation Protocol=reproduced2026.06 | 11 | — | — | — | — | — | — | — | — | — | — | 36 | 43 | 21 | 42 | 13 | |
| UI-TARS-2Model Category=GUI-Specialized Models2026.06 | 10 | — | — | — | — | — | — | — | — | — | — | 53.1 | 72 | 29 | 49 | 32 | |
| Qwen3-VL-8B-Instruct#Params=8B, Backbone=Qwen3-VL-8B backbone, Model Category=Backbone-Matched Models, Evaluation Protocol=reproduced2026.06 | 10 | — | — | — | — | — | — | — | — | — | — | 32.7 | 39 | 15 | 40 | 14 | |
| Step-GUI-8B#Params=8B, Backbone=Qwen3-VL-8B backbone, Model Category=Backbone-Matched Models2026.06 | 9 | — | — | — | — | — | — | — | — | — | — | 40.2 | 45 | 18 | 55 | 18 | |
| UI-TARS-1.5-7B#Params=7B, Model Category=GUI-Specialized Models2026.06 | 8 | — | — | — | — | — | — | — | — | — | — | 29.6 | 35 | 19 | 37 | 8 | |
| TianXi-Action-7B#Params=7B, Model Category=GUI-Specialized Models2026.06 | 8 | — | — | — | — | — | — | — | — | — | — | 30.4 | 34 | 19 | 43 | 5 | |
| Qwen3-VL-30B-A3B-Instruct#Params=30B-A3B, Backbone=Qwen3-VL-30B-A3B backbone, Model Category=Backbone-Matched Models, Evaluation Protocol=reproduced2026.06 | 8 | — | — | — | — | — | — | — | — | — | — | 31.9 | 43 | 15 | 37 | 11 | |
| Agent S w/ Claude-3.5-SonnetStep=152026.06 | — | — | — | — | — | — | — | — | — | — | 20.5 | — | — | — | — | — | |
| Agent S w/ GPT-4oStep=152026.06 | — | — | — | — | — | — | — | — | — | — | 20.6 | — | — | — | — | — | |
| Agent S2 w/ Claude-3.7-SonnetStep=502026.06 | — | — | — | — | — | — | — | — | — | — | 34.5 | — | — | — | — | — | |
| Agent S2 w/ Gemini-2.5-ProStep=502026.06 | — | — | — | — | — | — | — | — | — | — | 41.4 | — | — | — | — | — | |
| Aguvis-72B w/ GPT-4oStep=152026.06 | — | — | — | — | — | — | — | — | — | — | 17 | — | — | — | — | — | |
| Aria-UI w/ GPT-4oStep=152026.06 | — | — | — | — | — | — | — | — | — | — | 15.2 | — | — | — | — | — | |
| Claude 3.7 SonnetStep=1002026.06 | — | — | — | — | — | — | — | — | — | — | 28 | — | — | — | — | — | |
| GPT-5.5Model Category=General-Purpose Models2026.06 | — | — | — | — | — | — | — | — | — | — | — | 78.7 | — | — | — | — | |
| GUI-Owl-32BStep=1002026.06 | — | — | — | — | — | — | — | — | — | — | 19 | — | — | — | — | — | |
| GUI-Owl-32B + Ours (failure-case loop)Step=1002026.06 | — | — | — | — | — | — | — | — | — | — | 21.3 | — | — | — | — | — | |
| Holo2-30B-A3B#Params=30B-A3B, Backbone=Qwen3-VL-30B-A3B backbone, Model Category=Backbone-Matched Models2026.06 | — | — | — | — | — | — | — | — | — | — | — | 37.4 | — | — | — | — | |
| Jedi-7B w/ GPT-4oStep=1002026.06 | — | — | — | — | — | — | — | — | — | — | 27 | — | — | — | — | — | |
| Lite-GUI-30B-A3B#Params=30B-A3B, Backbone=Qwen3-VL-30B-A3B backbone, Model Category=Backbone-Matched Models2026.06 | — | — | — | — | — | — | — | — | — | — | — | 22.7 | — | — | — | — | |
| OpenAI CUA 4oStep=2002026.06 | — | — | — | — | — | — | — | — | — | — | 38.1 | — | — | — | — | — | |
| OpenAI CUA o3Step=2002026.06 | — | — | — | — | — | — | — | — | — | — | 42.9 | — | — | — | — | — | |
| OpenCUA-32BStep=1002026.06 | — | — | — | — | — | — | — | — | — | — | 34.5 | — | — | — | — | — | |
| OpenCUA-32B + Ours (failure-case loop)Step=1002026.06 | — | — | — | — | — | — | — | — | — | — | 38.2 | — | — | — | — | — | |
| OpenCUA-72BStep=1002026.06 | — | — | — | — | — | — | — | — | — | — | 42.3 | — | — | — | — | — | |
| OpenCUA-72B + Ours (Main)Step=1002026.06 | — | — | — | — | — | — | — | — | — | — | 48.9 | — | — | — | — | — | |
| OSExpertModel Category=Domain Specialized CUA Model, Max inference steps=50, Number of trials=32026.05 | — | — | 30.8 | 44.7 | 42.6 | 34.7 | — | — | — | — | — | — | — | — | — | — | |
| SEAgentModel Category=Domain Specialized CUA Model, Max inference steps=50, Number of trials=32026.05 | — | — | 42.3 | — | 22.7 | 31.8 | — | — | 35.3 | 40.5 | — | — | — | — | — | — | |
| UI-TARS-1.5Step=1002026.06 | — | — | — | — | — | — | — | — | — | — | 42.5 | — | — | — | — | — | |
| UI-TARS-1.5-7BStep=1002026.06 | — | — | — | — | — | — | — | — | — | — | 26.9 | — | — | — | — | — | |
| UI-TARS-72B-DPOStep=152026.06 | — | — | — | — | — | — | — | — | — | — | 22.7 | — | — | — | — | — | |
| UI-TARS-72B-DPOStep=502026.06 | — | — | — | — | — | — | — | — | — | — | 24.6 | — | — | — | — | — | |
| UI-TARS-72B-SFTStep=502026.06 | — | — | — | — | — | — | — | — | — | — | 18.8 | — | — | — | — | — |