ARC
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
ARC Easy
64.65Accuracy
4
ARC Challenge
34.04Accuracy
4
Translated ARC-Easy (test)
94.8Accuracy (English)
4
ARC Challenge
37.2Accuracy
4
ARC Easy
66.12Accuracy
4
ARC Diff
0.34Kendall Correlation
4
ARC
1.61MAE
3
ARC Challenge
23.61Zero-shot Accuracy
3
ARC Challenge
86.1Accuracy
3
ARC Easy
64.1Accuracy
3
ARC
0.664Kendall's Tau
3
ARC Easy
49.45Accuracy (ARC Easy)
3
ARC Easy 30% Label Noise
3.196NLL
3
ARC Challenge 30% Label Noise
3.445NLL
3
ARC Easy 20% Label Noise
3.181NLL
3
ARC Challenge 20% Label Noise
3.44NLL
3
ARC Challenge 5-shot
43.6Accuracy
3
ARC-Easy 5-shot
72.9Accuracy
3
ARC Easy
79.38Accuracy
3
ARC Easy (full test val)
32.1Accuracy
3
ARC
0Risk
3
ARC Challenge (eval)
0Pathwise Violations (PathV)
3
ARC Easy
0.001RMSE
3
ARC Challenge
0.022RMSE
3
ARC-C
80.6Accuracy
3