AppWorld
Benchmarks
TaskDatasetSOTA resultTrendResults
AppWorld normal (test)
79.8Task Goal Completion (TGC)
8
AppWorld (test-challenge)
9.1Task Goal Completion (TGC)
8
AppWorld
27.6Success Rate
7
AppWorld
22.36Avg@8
7
AppWorld
18.46Average Completion Score @4
7
AppWorld
83.7TGC Success Rate (N)
7
AppWorld challenge (test)
98.3TGC
7
AppWorld Average
59.5Average Score
7
AppWorld
0.05Cost
7
AppWorld
78.8Max Accuracy
6
AppWorld (test)
86.2Easy Success Rate (SR)
6
AppWorld 14-agent pool
93.3Score
5
AppWorld
64.9Accuracy
5
AppWorld (Resampled)
66.1Accuracy (%)
4
AppWorld (Official)
60.1Accuracy
4
AppWorld Hard 63 tasks (test_normal)
31.7Accuracy
4
AppWorld Medium 48 tasks (test_normal)
43.8Accuracy
4
AppWorld Easy normal 57 tasks (test)
73.7Accuracy
4
AppWorld Average 168 tasks (test normal)
44Accuracy
4
AppWorld (test-n)
81.15TGC Average
4
AppWorld Challenge Qwen-2.5-32B (test)
51Average Task Completion Score
4
AppWorld Normal Qwen-2.5-32B (test)
75Average Task Completion Score
4
AppWorld Normal (test)
89.5Task Goal Success Rate
4
AppWorld (test challenge)
27Task Goal Completion (TGC)
3
AppWorld online mode
57.7Accuracy
3