ARC
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
ARC Easy
0.065Capability Delta (Δ)
3
ARC Challenge
83.1Accuracy
3
ARC Challenge
74.55Accuracy
3
ARC
31.71Normalized Accuracy
3
ARC-Challenge Spanish (test)
32.48Normalized Accuracy
3
ARC-Challenge In-Distribution Llama-3.1-8B (train test)
81.79Accuracy
3
ARC Easy
73.86Accuracy
3
ARC Challenge
39.75Accuracy
3
ARC Challenge
63.5Normalized Log Accuracy
3
ARC Easy
87.5Normalized Log Accuracy
3
ARC Easy DE 25-shot (test)
80.1Normalized Log Accuracy
3
ARC Challenge Hungarian
39.73Accuracy
3
ARC Challenge Turkish
36.55Accuracy
3
ARC
14.03Average Relative Improvement
3
ARC-c OpenR1-Math Harder
70.6Accuracy
3
ARC Indonesian
24Accuracy
3
ARC Challenge Multilingual (test)
58.2ARC Accuracy (French)
3
ARC Hard 2
6.67P@1
2
ARC 1 (Standard)
50.62P@1
2
ARC Challenge
88.14Accuracy
2
ARC Challenge
45E*S
2
ARC Challenge
36.77Accuracy
2
ARC Easy
51.56Accuracy
2
ARC Easy
76Score
2
ARC Challenge
50.9Score
2