GSM8K
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
GSM8K 776 questions (test)
0Cost ($)
9
GSM8K (test)
1TPF (Time Per First Token)
9
GSM8K
74.3Acc (128)
9
GSM8K (test)
82.34Accuracy
9
GSM8K
95.9avg@3 Accuracy
9
GSM8K
84.6Accuracy
9
GSM8K (test)
32.2Pass@1
9
GSM8K
86.5Pass@1 Accuracy
9
GSM8K
95.72Pass@1
9
GSM8K (test)
90.52Accuracy
9
GSM8K
79.3Acceptance Rate
9
GSM8K (test)
87.3Accuracy
8
GSM8K (test)
1.25Perplexity (PPL)
8
GSM8K
94.8Accuracy
8
GSM8K (test)
61.03Pass@1
8
GSM8K
95.88Pass@1
8
GSM8K 500-example (test)
84.8Pass@1 Final
8
GSM8K (test)
62.7GSM8K Accuracy
8
GSM8K
71.5Expert Count
8
GSM8K (test)
38.36Exact Match Accuracy
8
GSM8K
84.9Pass@1 Accuracy
8
GSM8K
59.4Accuracy (Exact Match)
8
GSM8K synthetic variants short-prompt (val)
26.5Peak Memory (GB)
8
GSM8K
96.02Accuracy
8
GSM8K In-Domain
65.88Accuracy (GSM8K ID)
8