Mathematical Reasoning on GSM8K (test) (Accuracy, Throughput)
70.7AccuracyLLaDa-8B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LLaDa-8BPre-training dataset=proprietary data, Supervised fine-tuning (SFT)=true, Batch size=1, Context length=2562026.02 | 70.7 | — | |
| Duo-1.7BPre-training dataset=Nemotron-Pre-Training Dataset, Number of training tokens=2.1T, Supervised fine-tuning (SFT)=true, Batch size=1, Context length=2562026.02 | 65.8 | 24.6 | |
| AR-1.7BPre-training dataset=Nemotron-Pre-Training Dataset, Number of training tokens=2.1T, Supervised fine-tuning (SFT)=true, KV caching=true, Batch size=1, Context length=2562026.02 | 62.9 | 25.6 | |
| MDLM-1.7BPre-training dataset=Nemotron-Pre-Training Dataset, Number of training tokens=2.1T, Supervised fine-tuning (SFT)=true, Batch size=1, Context length=2562026.02 | 58.8 | 24.6 | |
| SMDM-1BPre-training dataset=slim-pajama, Supervised fine-tuning (SFT)=true, Batch size=1, Context length=2562026.02 | 58.5 | — | |
| Eso-LM-1.7BPre-training dataset=Nemotron-Pre-Training Dataset, Number of training tokens=2.1T, Supervised fine-tuning (SFT)=true, Batch size=1, Context length=2562026.02 | 33.4 | 25.6 |