GSM8K
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
GSM8K
89.82TPS
26
GSM8K
95.5Accuracy
26
GSM8K
61.56GSM8K 8-shot Accuracy
25
GSM8K
83.32CACC
25
GSM8K 8-shot (test)
93.5Accuracy
25
GSM8K
97.22Relative Overall Score
25
GSM8K
94.11Accuracy
25
GSM8K
80.64Accuracy
24
GSM8K
96.2Accuracy
24
GSM8K
85.22HH Safety Score
24
GSM8K
97.12maj@16 Accuracy
24
GSM8K
92.1Accuracy
24
GSM8k
66.42Average Accuracy
24
GSM8K
2.91Speedup
24
GSM8k Hard
7.3Average Forward Passes
24
GSM8k
6Avg Forward Passes
24
GSM8K
0False Positive Rate
24
GSM8K
92.4Accuracy
24
GSM8K
92.6Accuracy
24
GSM8K (test)
83.02Accuracy
24
GSM8K (test)
93.33Top-1 Accuracy
24
GSM8K zero-shot (test)
89.46Accuracy
24
GSM8K (test)
32Accuracy
24
GSM8K 10 (test)
66.4m1@t1
24
GSM8K Prolog (test)
89.3Pass@4
23