ARC
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
ARC-Challenge (In-Distribution)
82.55Accuracy
9
ARC-C
72.44Score
9
ARC Community
63.04Accuracy
9
ARC Challenge
66.38ARC-C Utility
9
ARC-C
97.7Score
9
ARC-C
95.5Accuracy
9
ARC-E
98.7Accuracy
9
ARC Challenge
95.9Accuracy
9
ARC-E
99.4Accuracy (ARC-E)
9
ARC (train)
26Tasks Solved
9
ARC
42.83Accuracy (ARC-C)
9
ARC zero-shot
50.96Accuracy (ARC zero-shot)
9
ARC Challenge
89.9Accuracy
9
ARC Challenge
0.066Mean Improvement
9
ARC-E
1.3Accuracy Delta
9
ARC Easy
4.7Accuracy Delta
9
ARC Challenge
86.8AUROC
9
ARC 1.0 (test)
98Accuracy
9
ARC-E Small Shift (test)
78.9Accuracy
9
ARC-C Small Shift (test)
69.5Accuracy
9
ARC-E Smaller Dist. Shift 18
76.88Accuracy
9
ARC-C Smaller Dist. Shift 18
69.93Accuracy
9
ARC-C 200-sample
85Accuracy
8
ARC Challenge
117.39Exact Match (EM)
8
ARC-C
65.9Expert Count
8