Arithmetic
Benchmarks
TaskDatasetSOTA resultTrendResults
Arithmetic trace-level 60/20/20 (test)
92.4Repair Accuracy
16
arithmetic LLAMA-3.2-3B-INSTRUCT (test)
70.2Avg@16
10
Arithmetic 1.0 (test)
1.017CPR
9
Arithmetic (test)
93.2Avg@16
8
Arithmetic
88.2F1 (X)
7
Arithmetic
87.8M' F1 Mean
7
Arithmetic
1.09CPR
6
Arithmetic
97.8Accuracy
5
Arithmetic Addition
95.4CPR
4
Arithmetic
71.764Accuracy (Easy)
4
Arithmetic
0.995Peak Alin
4
Arithmetic (val)
83Accuracy
3
Arithmetic
273HQ Neuron Count
2
Arithmetic concept family
0Accuracy
2
Arithmetic concept family
65.7Top-1 Accuracy
1