ARC
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
ARC-C
1,163Latency (NoKV, ms)
5
ARC Challenge (val)
91.7Accuracy
5
ARC-Easy zero-shot n=200/task
71Accuracy
5
ARC (Abstraction and Reasoning Corpus)
68.2RSR (%)
5
ARC Challenging
15.65LLMcritic Calls
5
ARC Challenging
15.65LLMcritic Calls
5
ARC Easy
33.53Accuracy
5
ARC-C en
82.81Accuracy
5
ARC Challenge
7.4Accuracy Improvement (Δ)
5
ARC-LoRA (test)
0.32MAE
5
ARC-LoRA (val)
0.34MAE
5
ARC Challenge
91Accuracy
5
Private human-curated 177 ARC-style tasks (evaluation set)
55.93Pass@k
5
ARC Challenge (test)
42.3Accuracy
5
ARC Easy (test)
68.9Accuracy
5
ARC Challenge
22.53Accuracy
4
ARC 2 (test)
7.8Pass@2
4
ARC Easy
1.594CPR
4
Arc
80.4Top-1 Accuracy
4
ARC (8 held-out tasks)
80Success Rate
4
ARC Challenge
22.87Normalized Accuracy
4
ARC Easy
97.14Minimum Score
4
ARC-C
55.1Accuracy
4
ARC-Challenge 25-shot (test)
58.79Accuracy
4
ARC Challenge
49.98ARC Score
4