ARC-AGI
Benchmarks
TaskDatasetSOTA resultTrendResults
ARC-AGI 2
100Accuracy (Pass@2)
33
ARC-AGI 2 (test)
24.9Pass@2 Exact Match Accuracy
28
ARC-AGI 1
98Accuracy (Pass@2)
27
ARC-AGI 2
77.1Pass@2
24
ARC-AGI 2
60Accuracy
18
ARC-AGI 1
97Accuracy
18
ARC-AGI v1 (test)
98Accuracy
17
ARC-AGI
4.5Pass@1
16
ARC-AGI 2 (public evaluation)
100Pass@2
13
ARC-AGI v2 (test)
100Accuracy
11
ARC-AGI 2
33.6Accuracy
11
ARC-AGI 2 (Semi-private)
100Accuracy
10
ARC-AGI Public Training Set (Easy) (60 tasks)
0.41Total Cost
10
ARC-AGI 2
57.5General Reasoning Score
9
ARC-AGI 1
89.9Score
9
ARC-AGI 2
50Accuracy
9
ARC-AGI
58.2ARC-1 Score
9
ARC-AGI 1
51FMR
7
ARC-AGI public evaluation set V2
97.9Accuracy
6
ARC-AGI 1 (test)
47.5Pass@2
6
ARC-AGI 3 (public evaluation)
78.4Score (%)
5
ARC-AGI (test)
67Accuracy (ARC-AGI Test)
5
ARC-AGI-3 25 Public Games v2
0RHAE
4
ARC-AGI
3.54Frugality Index (Fp)
4
ARC-AGI 2 (test)
51Accuracy (ARC-AGI 2 Test)
4