AndroidWorld
Benchmarks
TaskDatasetSOTA resultTrendResults
AndroidWorld
80Success Rate
200
AndroidWorld (test)
1Average Success Rate
119
AndroidWorld
62Accuracy
70
AndroidWorld
76.9Overall Success Rate
68
AndroidWorld latest (test)
78.9Success Rate
48
AndroidWorld
71.6Overall Task Success Rate
45
AndroidWorld
82.05Success Rate
27
AndroidWorld (online)
48.7Task Success Rate
25
AndroidWorld
77.6Success Rate
25
AndroidWorld M
53.8L1 Score
22
AndroidWorld
82.05Success Rate
20
AndroidWorld 138 tasks (test)
71.1Success Rate
18
AndroidWorld (full)
82.79Success Rate (Easy)
16
AndroidWorld (Evaluation set 116 templates)
62.9Average Success Rate (SR)
16
AndroidWorld
57.8Success Rate (SR)
15
AndroidWorld
92.5Precision
14
AndroidWorld (test)
80.2Pass@1
14
AndroidWorld 116 tasks
75Task Success Rate (TSR)
10
AndroidWorld
62.28Success Rate
9
AndroidWorld
83.2Task Success Rate
7
AndroidWorld (AW) (Online)
25.1Success Rate
6
AndroidWorld core20 safe general tasks
11Success Count (out of 20)
4
AndroidWorld
70.7Score
4
AndroidWorld (AW)
43.2AUV
4
AndroidWorld (smallest held-out split)
87.1Precision
3