Mathematical Reasoning on Math lm-evaluation-harness (GSM8k, MBPP, MATH)
1.44GSM8k AccuracyTransformer
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| TransformerTotal parameters=918m, Non-embedding parameters=764m2026.01 | 1.44 | 0 | 0.08 | |
| Transformer + Spelling Bee EmbeddingsTotal parameters=918m, Non-embedding parameters=764m2026.01 | 1.21 | 0.2 | 0.12 |