ARC
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
ARC-E Corrupt Setting
60.9Accuracy (ARC-E Corrupt)
8
ARC-C Corrupt Setting
43.3Accuracy
8
ARC-E Clean Setting
84.6Accuracy
8
ARC Challenge
58.1Accuracy
8
ARC Easy
83.9Accuracy
8
ARC-C
0.804AUROC
8
ARC-C
92.42Accuracy (ARC-C avg.@8)
8
ARC Challenge (dev)
40.61Accuracy
8
ARC Easy 25-shot
92Normalized Log Accuracy
8
ARC EL
37.5Accuracy (ARC EL)
8
ARC EN
51.11Accuracy
8
ARC-C
87.04Accuracy
8
ARC-C-ZH
33.96Score
8
ARC-C
41.13Accuracy
8
ARC Challenge
46.7Sycophancy Rate
8
ARC (test)
67Recall@5
8
ARC C
0.969Train R^2
7
ARC Challenge 0-shot
90.11Accuracy
7
ARC-C
88.53Accuracy
7
ARC 2
29.4Solve Rate
7
ARC-TGI
39.1Active-cell Accuracy
7
ARC-E
81.7Accuracy
7
ARC
100AR%
7
ARC Challenge
82Utility
7
ARC Easy
97Utility
7