ARC
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
ARC Challenge
39.42Accuracy (30% Sparsity)
4
ARC Challenge
96.7Accuracy
4
ARC Challenge
27.8Accuracy
4
ARC Easy
0.725Accuracy
4
ARC-c
85.9EM
4
ARC Challenge
78.4Accuracy
4
ARC Easy
76.6Pass@1 Score
4
ARC Easy
29.3pass@1
4
ARC Easy DE
84.8Normalized Log Accuracy
4
ARC Challenge
5.729Causal Fidelity Score (CFS)
4
ARCx
72.4Accuracy
4
ARC
100Success Rate (SR)
4
ARC
90.87Performance
4
ARC
95Cumulative Score (CS)
4
ARC-C
88.1ARC-C Score
4
ARC 130
1.33Memory Consumption (MB)
4
ARC Hard
32.59Score
4
ARC Challenge (test)
28Normalized Accuracy
4
ARC Easy (test)
53Normalized Accuracy
4
ARC-E
46.8Accuracy (GPT-2-Small)
4
ARC-C
21.8Accuracy (GPT-2-Small)
4
ARC
95Accuracy
4
ARC-C 0-shot
55.2Accuracy
4
ARC Challenge
6Normalized PLL Score
4
ARC-C
80.29Exact Match
4