Agentic Performance on OSWorld Verified
70.8OS PerformanceClaude-4.5-Sonnet
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Claude-4.5-SonnetModel Category=Proprietary Models, Planner=OpenAI o3, Resolution=1920x10802025.11 | 70.8 | 72.6 | 61.4 | 63.3 | 49 | 62.9 | |
| OpenAI o3Model Category=Proprietary Models, Planner=OpenAI o3, Resolution=1920x10802025.11 | 62.5 | 14.5 | 21.4 | 38.8 | 16.5 | 23 | |
| GROUNDNEXT-3BModel Category=Open-source Models, Training Protocol=RL, Planner=OpenAI o3, Resolution=1920x10802025.11 | 62.5 | 47 | 55 | 73.5 | 36.5 | 50.6 | |
| OpenCUA-72BModel Category=Open-source Models, Planner=OpenAI o3, Resolution=1920x10802025.11 | 58.3 | 47 | 53.8 | 73.5 | 20.4 | 46.1 | |
| JEDI-7BModel Category=Open-source Models, Planner=OpenAI o3, Resolution=1920x10802025.11 | 50 | 46.1 | 61.9 | 75.5 | 35.3 | 51 | |
| Claude-4-SonnetModel Category=Proprietary Models, Planner=OpenAI o3, Resolution=1920x10802025.11 | 45.8 | 39.3 | 48.1 | 59.2 | 27.9 | 41.4 | |
| UI-TARS-250705Model Category=Proprietary Models, Planner=OpenAI o3, Resolution=1920x10802025.11 | 41.7 | 50.4 | 55.7 | 51 | 14.7 | 41.8 | |
| OpenCUA-7BModel Category=Open-source Models, Planner=OpenAI o3, Resolution=1920x10802025.11 | 41.7 | 22.2 | 37.1 | 49 | 9.3 | 27 | |
| Qwen3-VL-FlashModel Category=Proprietary Models, Planner=OpenAI o3, Resolution=1920x10802025.11 | 40.9 | 53.6 | 55.1 | 22 | 22 | 41.6 | |
| UI-TARS-72B-DPOModel Category=Open-source Models, Planner=OpenAI o3, Resolution=1920x10802025.11 | 37.5 | 19 | 34.6 | 63.3 | 8.3 | 27.1 | |
| UI-TARS-1.5-7BModel Category=Open-source Models, Planner=OpenAI o3, Resolution=1920x10802025.11 | 33.3 | 29.9 | 37.9 | 53.1 | 9.1 | 29.6 | |
| CUAModel Category=Proprietary Models, Planner=OpenAI o3, Resolution=1920x10802025.11 | 23.9 | 34.6 | 55.1 | 18.3 | 18.3 | 31.4 | |
| Qwen2.5-VL-72BModel Category=Open-source Models, Planner=OpenAI o3, Resolution=1920x10802025.11 | 16.7 | 4.3 | 6.4 | 2 | 3.2 | 5 | |
| Kimi-VL-A3BModel Category=Open-source Models, Planner=OpenAI o3, Resolution=1920x10802025.11 | 12.5 | 6 | 21.7 | 18.4 | 1.1 | 10.3 | |
| OpenCUA-A3BModel Category=Open-source Models, Planner=OpenAI o3, Resolution=1920x10802025.11 | 12.5 | 16.3 | 21.7 | 46.9 | 2.2 | 17.7 | |
| Qwen2.5-VL-32BModel Category=Open-source Models, Planner=OpenAI o3, Resolution=1920x10802025.11 | 8.3 | 1.7 | 6.4 | 6.1 | 2.2 | 3.9 |