ARC
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
ARC Challenge
62.3Accuracy
6
ARC-c
51.5Accuracy
6
ARC Easy
78.59Score
6
ARC Challenge DE
66.9Normalized Log Accuracy
6
ARC Challenge EN
51.11Accuracy
6
ARC
86.31Accuracy
6
Mini-ARC
52.03Success Rate
6
ARC-C EU20 (test)
34.56Accuracy
6
ARC-C French
37.47Score
6
ARC-Easy (test)
97.14Score
6
ARC Challenge (test)
92.92Score
6
ARC-1 sampled 15 reasoning task instances Chollet 2019
86.47Task 1 Accuracy
6
ARC
0.788Accuracy
6
ARC v1 (Eva.)
100Accuracy
6
ARC Infer. v1
100Accuracy
6
ARC-C (abundant)
65.6Accuracy
6
ARC-C (medium)
64.9Accuracy
6
ARC-C (scarce)
64.5Accuracy
6
ARC-E abundant
75.8Accuracy
6
ARC Challenge
50.9Accuracy (0-shot)
6
ARC Ensemble
16.4HD95 (mm)
6
ARC FLAIR
44HD95 (mm)
6
ARC T2w
39HD95 (mm)
6
ARC T1w
8.8HD95 (mm)
6
ARCPrize
50.94Score
5