Desktop automation on WindowsAgentArena (WAA) v1 (test)
61Overall ScoreVLAA-GUI
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| VLAA-GUIMax Steps=100, Backbone=Gemini 3 Flash2026.04 | 61 | 35 | 73.3 | 87.5 | 66.7 | 52.4 | 83.3 | |
| VLAA-GUIMax Steps=50, Backbone=Gemini 3 Flash2026.04 | 60.4 | 32.6 | 73.3 | 87.5 | 66.7 | 52.4 | 75 | |
| Agent S3Max Steps=100, Backbone=GPT-52026.04 | 56.6 | — | — | — | — | — | — | |
| VLAA-GUI - Loop BreakerMax Steps=100, Backbone=Gemini 3 Flash, Ablation=Remove Loop Breaker2026.04 | 55.8 | 30.2 | 66.7 | 83.3 | 58.3 | 47.6 | 83.3 | |
| Agent S3Max Steps=50, Backbone=GPT-52026.04 | 54.1 | — | — | — | — | — | — | |
| VLAA-GUI - Search AgentMax Steps=100, Backbone=Gemini 3 Flash, Ablation=Remove Search Agent2026.04 | 53.9 | 25 | 63.3 | 75 | 66.7 | 47.6 | 83.3 | |
| VLAA-GUI - Loop BreakerMax Steps=50, Backbone=Gemini 3 Flash, Ablation=Remove Loop Breaker2026.04 | 52.6 | 27.9 | 66.7 | 83.3 | 58.3 | 38.1 | 75 | |
| VLAA-GUI - Completeness VerifierMax Steps=50, Backbone=Gemini 3 Flash, Ablation=Remove Completeness Verifier2026.04 | 51.3 | 11.6 | 73.3 | 70.8 | 54.2 | 61.9 | 83.3 | |
| VLAA-GUI - Completeness VerifierMax Steps=100, Backbone=Gemini 3 Flash, Ablation=Remove Completeness Verifier2026.04 | 51.3 | 13.9 | 73.3 | 70.8 | 54.2 | 57.1 | 83.3 | |
| GTA1-32BMax Steps=100, Backbone=o32026.04 | 51.2 | — | — | — | — | — | — | |
| UI-TARS-2Max Steps=502026.04 | 50.6 | — | — | — | — | — | — | |
| VLAA-GUI - Search AgentMax Steps=50, Backbone=Gemini 3 Flash, Ablation=Remove Search Agent2026.04 | 49.4 | 18.6 | 63.3 | 75 | 66.7 | 38.1 | 66.7 | |
| Qwen3-VL-32BMax Steps=502026.04 | 31.7 | 19.1 | 49.7 | 54.2 | 21.1 | 42.2 | 25 |