GUI Automation on WindowsAgentArena
44.2Success RateHiViG
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| HiViGPolicy Backbone=Gemini-3-Flash2026.06 | 44.2 | — | — | — | — | — | — | — | — | |
| HiViGPolicy Backbone=Qwen3-VL-32B-Thinking2026.06 | 38 | — | — | — | — | — | — | — | — | |
| CGI (32B)Policy Backbone=Gemini-3-Flash2026.06 | 37.9 | — | — | — | — | — | — | — | — | |
| CGI (8B)Policy Backbone=Gemini-3-Flash2026.06 | 37.9 | — | — | — | — | — | — | — | — | |
| Base agentPolicy Backbone=Gemini-3-Flash2026.06 | 35.8 | — | — | — | — | — | — | — | — | |
| OpenCUAPolicy Backbone=Gemini-3-Flash2026.06 | 35.8 | — | — | — | — | — | — | — | — | |
| Base agentPolicy Backbone=Qwen3-VL-32B-Thinking2026.06 | 35.7 | — | — | — | — | — | — | — | — | |
| OpenCUAPolicy Backbone=Qwen3-VL-32B-Thinking2026.06 | 35.2 | — | — | — | — | — | — | — | — | |
| SE-WSMPolicy Backbone=Gemini-3-Flash2026.06 | 35.1 | — | — | — | — | — | — | — | — | |
| GUI-Critic-R1Policy Backbone=Qwen3-VL-32B-Thinking2026.06 | 34.4 | — | — | — | — | — | — | — | — | |
| CGI (32B)Policy Backbone=Qwen3-VL-32B-Thinking2026.06 | 33.7 | — | — | — | — | — | — | — | — | |
| GUI-Critic-R1Policy Backbone=Gemini-3-Flash2026.06 | 32.5 | — | — | — | — | — | — | — | — | |
| SE-WSMPolicy Backbone=Qwen3-VL-32B-Thinking2026.06 | 31.6 | — | — | — | — | — | — | — | — | |
| CGI (8B)Policy Backbone=Qwen3-VL-32B-Thinking2026.06 | 31 | — | — | — | — | — | — | — | — | |
| Agent SBase Model=GPT-4o2025.02 | — | 0 | 13.3 | 45.8 | 29.2 | 19.1 | 22.2 | — | — | |
| Agent S2Base Model=Claude-3.7-Sonnet2025.02 | — | 7 | 16.4 | 54.2 | 62.5 | 28.6 | 33.3 | — | — | |
| Agent S3Step=50, Backbone=GPT-52026.01 | — | — | — | — | — | — | — | — | 54.1 | |
| Agent S3Step=100, Backbone=GPT-52026.01 | — | — | — | — | — | — | — | — | 56.6 | |
| GPT-4o2025.02 | — | 0 | 13.7 | 29.2 | 0 | 10.3 | 0 | — | — | |
| GPT-4o-mini2025.02 | — | 0 | 14.9 | 8.3 | 0 | 0 | 0 | — | — | |
| NAVI2025.02 | — | 0 | 27.3 | 33.3 | 27.3 | 30.3 | 8.3 | — | — | |
| OS-SYMPHONYStep=50, Backbone=Qwen3-VL-32B-Inst.2026.01 | — | 26.19 | 46.33 | 75 | 47.37 | 27.9 | 41.67 | 69.23 | 45.32 | |
| OS-SYMPHONYStep=50, Backbone=GPT-5-Mini2026.01 | — | 42.86 | 73 | 79.17 | 68.42 | 48.66 | 66.67 | 69.23 | 62.15 | |
| OS-SYMPHONYStep=50, Backbone=GPT-52026.01 | — | 54.76 | 73 | 75 | 42.11 | 70.09 | 75 | 53.85 | 63.45 | |
| Phi3-V2025.02 | — | 0 | 6.9 | 8.3 | 0 | 6.2 | 0 | — | — | |
| Qwen3-VL-32B-InstructStep=502026.01 | — | 19.05 | 49.66 | 54.17 | 21.05 | 42.19 | 25 | 0 | 31.68 | |
| UI-TARS-1.5-7BStep=502026.01 | — | — | — | — | — | — | — | — | 42.1 | |
| UI-TARS-2Step=502026.01 | — | — | — | — | — | — | — | — | 50.6 | |
| WorldGUI-AgentBase Model=Claude-3.5-Sonnet2025.02 | — | 7 | 53.3 | 45.8 | 33.3 | 28.6 | 33.3 | — | — |