OSWorld
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
OSWorld VLC (test)
70.7Success Rate
4
OSWorld Chrome (test)
70.6Success Rate
4
OSWorld
44.1Success Rate
4
OSWorld (test)
25Chrome Success Rate
4
OSWORLD noisy (Overall)
29Pass@8 Success Rate
3
OSWORLD noisy (Held-out)
5.1Pass@8 Success Rate
3
OSWORLD noisy (Trained (86))
88.4Pass@8 Success Rate
3
OSWORLD clean (Overall)
30.3Pass@8 Success Rate
3
OSWORLD clean (held-out)
7.5Pass@8 Success Rate
3
OSWORLD clean (Trained (86))
87.2pass@8 Success Rate
3
OSWorld Online evaluation
57.1Accuracy (50 steps)
3
OSWorld
47.5Headline SR
3
OSWorld
47.4Score
3
OSWorld (All)
27Success Rate (All)
3
OSWorld 50-step
34.5Headline SR
2
OSWorld 30-step
10.9Feasible Success Rate
2
OSWorld 100-step
72.6Headline SR
1
OSWorld
42.5Accuracy
1