Mathematical Reasoning on GSM-8K (train)
0.056Train LossMGUP-AdamW
Evaluation results
| Method | Links | |
|---|---|---|
| MGUP-AdamWModel=LLaMA2-7B, Evaluation Protocol=Fine-tuning2026.06 | 0.056 | |
| MicroAdamWModel=LLaMA2-7B, Evaluation Protocol=Fine-tuning, m=102026.06 | 0.057 | |
| AdamWModel=LLaMA2-7B, Evaluation Protocol=Fine-tuning2026.06 | 0.064 | |
| AdamW-8bModel=LLaMA2-7B, Evaluation Protocol=Fine-tuning2026.06 | 0.069 | |
| GALoreModel=LLaMA2-7B, Evaluation Protocol=Fine-tuning, rank=5122026.06 | 0.07 | |
| LDAdamWModel=LLaMA2-7B, Evaluation Protocol=Fine-tuning, rank=5122026.06 | 0.073 | |
| C-AdamWModel=LLaMA2-7B, Evaluation Protocol=Fine-tuning2026.06 | 0.081 |