ARC
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
ARC Russian (test)
91Accuracy
11
ARC Spanish (test)
0.92Accuracy
11
ARC Portuguese (test)
92Accuracy
11
ARC Vietnamese (test)
88ARC Score
11
ARC Challenge
96.8Accuracy
11
ARC Challenge
0.47NLL
10
ARC-C
129Throughput (tokens/s)
10
ARC-E
84.68Accuracy
10
ARC Challenge
95.22Accuracy
10
ARC Easy
94.82Accuracy
10
ARC (eval)
75Tasks Solved
10
ARC-Challenge (first 300 questions)
34.89Average Accuracy
10
ARC Easy (first 300 questions)
42.33Average Accuracy
10
ARC-c
50.77Accuracy
10
ARC Challenge
66.3Accuracy
10
ARC Challenge Arabic
27.97Normalized Accuracy
10
ARC-Easy Arabic
28.98Acc (Norm)
10
ARC Challenge
90.3Accuracy
10
ARC Challenge (test)
95.7Accuracy
10
ARC-Challenge Chinese
35.67Score
10
ARC-Hard chat (Māori)
59.22Accuracy
10
ARC Easy
86.9Accuracy
10
ARC Challenge
70.1Accuracy
10
ARC Challenge (dev)
56.84Accuracy
10
ARC Easy (dev)
72.49Accuracy
10