ARC
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
ARC-Nuc
88AUROC
48
ARC-BC
85AUROC
48
ARC-Nuc
0.05E-AURC
48
ARC BC
14E-AURC
48
ARC-Challenge 0-shot (test)
90.4Accuracy
48
ARC Challenge
30Non-generative Accuracy
48
ARC-c (test)
91.3Accuracy
48
ARC-C
72.44Accuracy (ARC-C)
46
ARC Challenge
51.1Accuracy
46
ARC
94.2Accuracy
45
ARC Challenge
94.3Accuracy
44
ARC Easy
93.01Accuracy (After Attack)
44
Arc-Easy v1 (test)
83.21Accuracy
43
ARC Challenge
95.5P@1
43
ARC-C
69.71ARC-C Score
43
ARC-Easy (ARC-E) 0-shot
79.5Accuracy
43
ARC-e
82.55Accuracy (%)
39
ARC Challenge
76.1Character-level Accuracy
37
ARC-E 0-shot
87.88Accuracy
37
ARC-C
65.27Accuracy
36
ARC Challenge
64.1ARC-C Score
36
ARC Easy
85.4Accuracy (ARC-E)
36
ARC-E
79.38Zero-shot Accuracy
36
ARC Challenge
48.46Zero-shot Accuracy
36
ARC Easy
78.3Accuracy
36