ARC
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
TinyARC
33.49Accuracy
2
Welsh ARC-Easy
72Accuracy
2
ARC Challenge (200 samples)
0.525Log-likelihood Score
2
ARC Easy (200 samples)
0.515Log-likelihood Score
2
ARC Distributed System
100Latency (ms)
2
ARC Easy
1.237BPC
2
ARC Challenge
1.028BPC
2
ARC Challenge
1.203Alpha
2
ARC Challenge
52.14Accuracy
2
ARC Challenge DE
57.2Normalized Log Accuracy
2
ARC
75.1Accuracy
2
ARC-C
25.34Score
2
ARC-C
321LEARNED Count
2
ARC-C
0.007MAE
2
ARC (Abstraction and Reasoning Corpus)
903Speedup
2
ARC Easy
20Attack Success Rate (Before Mitigation)
2
ARC-Challenge (ARC-C)
0.7Delta vs Baseline (pp)
1
ARC-c
41.47Accuracy (ARC-c)
1
ARC-e
70.18Accuracy
1
ARC
—Primary metric
0
ARC-E 1.0 (test)
—Primary metric
0
ARC-C 1.0 (test)
—Primary metric
0
ARC Easy 72 (test)
—Score
0