ARC
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
ARC Easy
87.8Accuracy (Character-level)
20
ARC (test)
1.165MAE (%)
20
ARC Challenge (test)
0.921Spearman Rho
20
ARC Challenge
97.95Attack Success Rate (ASR)
20
ARC Challenge
66.1Accuracy (ARC)
19
ARC-Challenge 25-shot
0.637Accuracy
19
ARC Challenge 25-shot (test)
69.8Accuracy
19
ARC-Easy (val)
96.2Accuracy
19
ARC-C
93.6Accuracy (ARC-C)
18
ARC Easy
81.1Score
18
ARC-C
86Accuracy
18
ARC-C
1.11CPR Faithfulness Score
18
ARC-E
1.06CPR Faithfulness Score
18
ARC Challenge 25-shot
61.2Accuracy
18
ARC Challenge
94.1Pass@1
18
ARC Challenge (test)
57.8Accuracy
18
ARC randomly sampled 400 instances (test)
84.8Accuracy
18
ARC Challenge
89.4Accuracy
17
ARC Challenge (test)
87.3Accuracy
17
ARC Challenge
73.89Score
17
ARC-C
43.35Accuracy (Zero-shot)
17
ARC-C
78.8Accuracy
17
ARC-Challenge
46.93Normalized Accuracy
17
ARC Challenge
77.3Capability
16
ARC Mini
97.32Accuracy
16