OS GUI Agentic Task Execution on OSWorld 361 tasks (Verified)
79.17OS Success RateGTA1 w/ GPT-5
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| GTA1 w/ GPT-5Step=1002026.01 | 79.17 | 63.91 | 62.56 | 79.59 | 50.91 | 63.41 | |
| OS-SYMPHONY w/ GPT-5Step=1002026.01 | 79.17 | 65.73 | 67.76 | 69.23 | 57.98 | 65.84 | |
| Agent S3 w/ GPT-5Step=1002026.01 | 77.5 | 66.46 | 61.23 | 69.8 | 51.37 | 62.63 | |
| CoAct-1 w/ GPT-5Step=1002026.01 | 75 | 62.93 | 57.94 | 71.43 | 47.87 | 59.93 | |
| CoAct-1 w/ GPT-5Step=1502026.01 | 75 | 62.93 | 61.78 | 71.43 | 47.87 | 60.76 | |
| OS-SYMPHONY w/ GPT-5Step=502026.01 | 75 | 64.85 | 61.19 | 69.23 | 54.86 | 63.61 | |
| UiPath w/ GPT-5Step=502026.01 | 73.91 | 49.52 | 62.12 | 71.43 | 37.3 | 53.69 | |
| OS-SYMPHONY w/ GPT5-MiniStep=502026.01 | 73.68 | 58.17 | 61.39 | 75 | 47.37 | 58.05 | |
| Claude-Sonnet-4.5Step=1002026.01 | 70.83 | 72.59 | 61.35 | 63.27 | 49.54 | 62.84 | |
| CoAct-1 w/ GPT-5Step=502026.01 | 70.83 | 60.65 | 54.09 | 69.39 | 42.37 | 56.39 | |
| OS-SYMPHONY w/ Qwen3-VL-32B-ThinkingStep=502026.01 | 70.83 | 40.97 | 66.04 | 73.08 | 31.22 | 50.23 | |
| DeepMiner-Mano-72BStep=1002026.01 | 66.67 | 63.22 | 52.51 | 83.67 | 24.41 | 53.91 | |
| Claude 4.5 SonnetParams=-2026.03 | 62.9 | — | — | — | — | — | |
| Agent S3 w/ GPT-5-MiniStep=50, reproduced_by_paper=true2026.01 | 62.5 | 54.62 | 46.67 | 44.9 | 37.04 | 47.58 | |
| OpenCUA-72BStep=1002026.01 | 61.13 | 44.73 | 49.95 | 72.58 | 22.16 | 44.91 | |
| OS-SYMPHONY w/ Qwen3-VL-32B-InstructStep=502026.01 | 58.33 | 40.94 | 53.54 | 75.1 | 31.24 | 46.86 | |
| Agent S2.5 w/ o3Params=7B w/ -2026.03 | 56 | — | — | — | — | — | |
| GTA1-32B w/ o3Params=32B w/-2026.03 | 55.4 | — | — | — | — | — | |
| UI-TARS-2Params=-2026.03 | 53.1 | — | — | — | — | — | |
| GTA1-7B w/ o3Params=7B w/-2026.03 | 53.1 | — | — | — | — | — | |
| Jedi-7B w/ o3Params=7B w/ -2026.03 | 50.2 | — | — | — | — | — | |
| DeepMiner-Mano-7BStep=1002026.01 | 50 | 39.28 | 44.87 | 73.47 | 17.2 | 40.15 | |
| Agent S3 w/ Qwen3-VL-32B-InstructStep=50, reproduced_by_paper=true2026.01 | 50 | 36.67 | 50.62 | 61.22 | 21.96 | 40.11 | |
| Agent S2 w/ GPT-5Params=7B w/ -2026.03 | 48.8 | — | — | — | — | — | |
| Qwen3-VL-32B-Thinking w/ GPT-4.1Params=32B w/ -2026.03 | 43.2 | — | — | — | — | — | |
| UI-TARS-1.5Params=-2026.03 | 41.8 | — | — | — | — | — | |
| UI-TARSStep=1002026.01 | 41.67 | 50.42 | 55.69 | 51.02 | 14.66 | 41.85 | |
| UI-TARS-2Step=1002026.01 | 41.67 | 61.11 | 62.12 | 61.22 | 34.13 | 53.1 | |
| Claude 4 SonnetParams=-2026.03 | 41.4 | — | — | — | — | — | |
| Qwen3-VL-32B-Thinking w/ TraceR1Params=32B w/ 8B2026.03 | 41.2 | — | — | — | — | — | |
| OpenAI CUA-o3Params=-2026.03 | 38.1 | — | — | — | — | — | |
| Qwen3-VL-235B-A22B-ThinkingParams=236B2026.03 | 38.1 | — | — | — | — | — | |
| Seed1.5-VLParams=-2026.03 | 36.7 | — | — | — | — | — | |
| Qwen3-VL-32B-ThinkingParams=32B2026.03 | 35.6 | — | — | — | — | — | |
| OpenCUA-32BParams=32B2026.03 | 34.8 | — | — | — | — | — | |
| Qwen3-VL-8B-ThinkingParams=8B2026.03 | 33.9 | — | — | — | — | — | |
| UI-TARS-1.5-7B w/ GPT-4.1Params=7B w/ -2026.03 | 31.6 | — | — | — | — | — | |
| UI-TARS-1.5-7B w/ TraceR1Params=7B w/ 8B2026.03 | 30.9 | — | — | — | — | — | |
| UI-TARS-1.5-7BParams=7B2026.03 | 27.4 | — | — | — | — | — | |
| UI-TARS-72B-DPOParams=72B2026.03 | 27.1 | — | — | — | — | — | |
| OpenCUA-7BParams=7B2026.03 | 26.6 | — | — | — | — | — | |
| o3Params=-2026.03 | 23 | — | — | — | — | — | |
| Qwen2.5-vl-72bParams=72B2026.03 | 5 | — | — | — | — | — | |
| Qwen3-VL-32B-InstructStep=502026.01 | — | — | — | — | — | 32.4 | |
| Qwen3-VL-32B-ThinkingStep=502026.01 | — | — | — | — | — | 41 |