AppWorld
Benchmarks
TaskDatasetSOTA resultTrendResults
AppWorld normal (test)
91.2Task Goal Completion
37
AppWorld (test challenge)
32Goal Completion Score
35
AppWorld Normal (test)
92.3TGC
29
AppWorld
90TGC
28
AppWorld C (test)
48.4PR Score
26
AppWorld N (test)
57.1PR Score
26
AppWorld
63.6Avg@4
25
AppWorld (test-normal)
56.5Accuracy
22
AppWorld Challenge (test)
84.2Task Goal Completion (TGC)
21
AppWorld (Challenge)
83.7TGC
20
AppWorld Normal
89.3Task Goal Completion (TGC)
20
AppWorld 1.0 (test_challenge)
95.4SGC
19
AppWorld 1.0 (test_normal)
97.2Sub-Goal Completion Rate
19
AppWorld
64.88Avg@4
19
AppWorld normal (test)
4,554Avg@8 Success
15
AppWorld Challenge (test)
49.88Task Goal Completion (TGC)
13
AppWorld LeaderBoard
48.8Greedy Success Rate
13
AppWorld normal (test)
98.2Task Goal Completion (TGC)
12
AppWorld (test-normal)
45.6Task Goal Completion (TGC)
11
AppWorld normal Hard (test)
39.7Accuracy
11
AppWorld Easy normal (test)
86Accuracy
11
AppWorld
22.6Success Rate
10
AppWorld-Challenge
52.8Task Goal Completion (TGC)
10
AppWorld Normal
71.4Task Goal Completion (TGC)
10
AppWorld v1.0 (test)
0.71R-precision (AZ)
9