GSM8K
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
GSM8K
88.1Accuracy (Before GSM8K)
4
GSM8K (test)
1,162Pass Count
4
GSM8K
96.36Accuracy
4
GSM8K
3.25Success Rate (SR)
4
GSM8K
74.76Accuracy (128 tokens)
4
GSM8K
80REST
4
GSM8K
4.42Acceptance Length
4
GSM8K
5.73Acceptance Length
4
GSM8K n=1,319 (full set)
91.43Accuracy
4
GSM8K n=200 (pilot)
94Accuracy
4
GSM8K
94.8Accuracy
4
GSM8K
420.7Throughput (tok/s)
4
GSM8K (test)
55.9Accuracy
4
50 GSM8K-style problems
78Accuracy
4
GSM8K
91.1ASR
4
GSM8K
70.8Accuracy (Ref)
4
GSM8K
85Pass@1 Rate
4
GSM8K
77.9Accuracy
4
GSM8K (test)
96.3Accuracy
4
GSM8K
43.6Pass@16 Rate
4
GSM8K
78.2Compression Accuracy
4
GSM8K
46Exact Match Accuracy
4
GSM8k OOD (test)
3.15ECE
4
GSM8K
0.26Distinct-1
4
GSM8K (test)
92.4Accuracy
4