GAIA
Benchmarks
TaskDatasetSOTA resultTrendResults
GAIA
93.2Accuracy
291
GAIA (val)
96.23Level 1 Score
97
GAIA
80.61Avg Performance
77
gaia
81.9Accuracy
59
GAIA
28.12Accuracy
50
GAIA
83.4Pass@1 Score
38
GAIA
84.5Accuracy
36
GAIA
71.5Task Success Rate
30
GAIA (test)
41.7Unreliability Rate
28
GAIA text
69.9Average Accuracy
28
GAIA
33.3Internalized Inference Rate
26
GAIA
78.49Level 1 Score
26
GAIA (test)
84.5Pass@1
25
GAIA
1.6Execution Time (min)
25
GAIA text-only (val)
70.9Accuracy
24
GAIA
78.2Accuracy
24
GAIA n=165 (dev)
73.93Average Accuracy
23
GAIA
70.5Success Rate
23
GAIA
88.5Avg@8 Score
22
GAIA
51Accuracy (Pass@4)
22
GAIA
0.672Accuracy
21
GAIA text-only original (test)
74.1Pass@1
20
GAIA Text
76.4Accuracy
19
GAIA text-only
81.9Score
19
GAIA text
70.5GAIA Average Score
19