UI Agent Evaluation on OSWorld
40.3SR (15 Steps)Qwen-3-VL-32B
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Qwen-3-VL-32BHistory=5, Drop=No2026.05 | 40.3 | 9.1 | 42.8 | 18.9 | 42.5 | 27.3 | 15,229 | |
| Qwen-3-VL-32BHistory=3, Drop=No2026.05 | 39.8 | 9.3 | 42.7 | 19.6 | 42.4 | 27.8 | 11,346 | |
| Qwen3-VL-30B-A3BHistory=5, Drop=No2026.05 | 35.8 | 9.3 | 40.4 | 22.2 | 39.2 | 29.6 | 15,198 | |
| Qwen3-VL-30B-A3BHistory=3, Drop=No2026.05 | 35.2 | 9.5 | 39.8 | 22.6 | 38.6 | 30 | 11,318 | |
| Qwen-3-VL-32BHistory=1, Drop=No2026.05 | 33.2 | 10.4 | 41.1 | 21.4 | 41 | 28.4 | 4,133 | |
| ReVisionHistory=5, Drop=Yes2026.05 | 32.1 | 8.3 | 35.9 | 19.8 | 34 | 26.4 | 6,963 | |
| Qwen-3-VL-8BHistory=5, Drop=No2026.05 | 30.9 | 11.6 | 35.6 | 19.2 | 35.3 | 25.8 | 15,171 | |
| OpenCUAHistory=5, Drop=No2026.05 | 30.8 | 10 | 34.4 | 23.6 | 32.5 | 30.9 | 15,075 | |
| ReVisionHistory=5, Drop=No2026.05 | 30.7 | 9.7 | 34.5 | 22.7 | 32.3 | 29.7 | 15,071 | |
| ReVisionHistory=3, Drop=No2026.05 | 30.6 | 9.8 | 34.1 | 22.9 | 32.2 | 29.8 | 11,078 | |
| OpenCUAHistory=3, Drop=No2026.05 | 30.5 | 10.1 | 34.1 | 23.8 | 32.2 | 31.1 | 11,193 | |
| Qwen-3-VL-8BHistory=3, Drop=No2026.05 | 30.2 | 11.9 | 35.1 | 19.8 | 34.8 | 26.2 | 11,284 | |
| Qwen-3-VL-8BHistory=1, Drop=No2026.05 | 29.8 | 12.8 | 34.2 | 20.7 | 33.9 | 27.3 | 4,061 | |
| ReVisionHistory=3, Drop=Yes2026.05 | 28.6 | 8.9 | 32 | 21 | 30.5 | 27.9 | 5,074 | |
| Qwen3-VL-30B-A3BHistory=1, Drop=No2026.05 | 27.9 | 10.6 | 32.4 | 23.6 | 31 | 30.8 | 4,117 | |
| UI-TARS-1.5-7BHistory=5, Drop=No2026.05 | 26 | 10.7 | 29.3 | 24.3 | 28.8 | 28.2 | 15,141 | |
| UI-TARS-1.5-7BHistory=3, Drop=No2026.05 | 25.5 | 10.9 | 28.5 | 24.6 | 28.5 | 28.6 | 11,227 | |
| UI-TARS-1.5-7BHistory=1, Drop=No2026.05 | 24.1 | 11.4 | 27.3 | 25.2 | 27.1 | 29.1 | 4,058 | |
| OpenCUAHistory=1, Drop=No2026.05 | 23.8 | 11.2 | 27.4 | 24.6 | 26 | 31.8 | 3,994 | |
| UI-TARS-72B-DPOHistory=5, Drop=No2026.05 | 21.1 | 11.4 | 24.9 | 24.6 | 24.6 | 31.4 | 15,032 | |
| UI-TARS-72B-DPOHistory=3, Drop=No2026.05 | 20.9 | 11.7 | 24.4 | 25 | 24.2 | 32.6 | 11,159 | |
| UI-TARS-72B-DPOHistory=1, Drop=No2026.05 | 20.7 | 12.2 | 23.7 | 25.6 | 23.5 | 32.9 | 3,967 | |
| Kimi-VL-A3BHistory=5, Drop=No2026.05 | 9.5 | 11.6 | 10.2 | 25.6 | 10.9 | 33.1 | 15,136 | |
| Kimi-VL-A3BHistory=3, Drop=No2026.05 | 9.2 | 11.8 | 10 | 25.9 | 10.7 | 33.5 | 11,221 | |
| Kimi-VL-A3BHistory=1, Drop=No2026.05 | 8.5 | 12.6 | 9.7 | 26.5 | 10.3 | 34.1 | 4,094 | |
| Qwen2.5-VL-72BHistory=5, Drop=No2026.05 | 4.1 | 12.4 | 4.9 | 27 | 5.5 | 35 | 15,183 | |
| Qwen2.5-VL-72BHistory=3, Drop=No2026.05 | 3.9 | 12.6 | 4.7 | 27.3 | 5.3 | 35.4 | 11,307 | |
| Qwen2.5-VL-72BHistory=1, Drop=No2026.05 | 3.6 | 13.4 | 4.4 | 28.1 | 5 | 36.2 | 4,109 | |
| Qwen2.5-VL-32BHistory=5, Drop=No2026.05 | 3 | 12.5 | 3.8 | 27 | 4.4 | 35.2 | 15,087 | |
| Qwen2.5-VL-32BHistory=3, Drop=No2026.05 | 2.9 | 12.7 | 3.6 | 27.4 | 4.2 | 35.6 | 11,238 | |
| Qwen2.5-VL-32BHistory=1, Drop=No2026.05 | 2.5 | 13.8 | 3.3 | 28.5 | 3.9 | 36.7 | 4,067 | |
| Qwen2.5-VL-7BHistory=5, Drop=No2026.05 | 2.3 | 13.2 | 2.7 | 28.1 | 3.5 | 36.5 | 15,062 | |
| Qwen2.5-VL-7BHistory=3, Drop=No2026.05 | 1.9 | 13.6 | 2.5 | 28.4 | 3.2 | 37.1 | 11,176 | |
| Qwen2.5-VL-7BHistory=1, Drop=No2026.05 | 1.6 | 14.5 | 2.2 | 29.6 | 2.7 | 38.2 | 4,013 |