ARC
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
ARC Easy
86.4Accuracy
108
ARC Challenge
59Normalized Accuracy
105
ARC Challenge (test)
91.2Accuracy
103
ARC Challenge
97.2Accuracy
100
ARC-Easy zero-shot
97.43Zero-shot Accuracy
94
ARC
98.8ARC Accuracy
82
ARC Challenge
96.7Accuracy
81
ARC Challenge (val)
93.3Accuracy
76
ARC-Easy (test)
97.2Accuracy
74
ARC Challenge (test)
84.1Accuracy
71
ARC-c
96.82Accuracy
69
ARC-Easy (test)
89.1Accuracy
68
ARC-C
86Accuracy (ARC-C)
67
ARC Easy
78.2Zero-shot Acc (ARC Easy)
67
ARC-C
97Accuracy
65
ARC Challenge
89.6Accuracy
63
ARC
93Accuracy
61
ARC Challenge (test)
94.3Accuracy
60
ARC-e (test)
96.07Accuracy
60
ARC-E
81.1Normalized Accuracy (ARC-E)
59
ARC Challenge
0.946Accuracy (ARC)
56
ARC Challenge
20.99Accuracy
56
ARC Easy
84.1Normalized Accuracy
55
ARC Challenge
60.41Zero-Shot Accuracy
53
ARC Challenge
93.7Accuracy
50