Mathematical Reasoning on GSM8K (test) (Pass@1)
86.4Pass@1LEMMA (w/ MetaMath)
Evaluation results
| Method | Links | |
|---|---|---|
| LEMMA (w/ MetaMath)Backbone=LLaMA3-8B, # Samples=403.59k2025.03 | 86.4 | |
| LEMMA (w/ MetaMath)Backbone=DeepSeekMath-7B, # Samples=403.59k2025.03 | 83 | |
| S³C-Math (w/ MetaMath)Backbone=LLaMA3-8B, # Samples=927k2025.03 | 82.9 | |
| S³C-Math (w/ MetaMath)Backbone=DeepSeekMath-7B, # Samples=927k2025.03 | 82.5 | |
| LEMMABackbone=DeepSeekMath-7B, # Samples=88.90k2025.03 | 80.4 | |
| MetaMathBackbone=DeepSeekMath-7B, # Samples=394.99k2025.03 | 79.4 | |
| MetaMathBackbone=LLaMA3-8B, # Samples=394.99k2025.03 | 79.2 | |
| LEMMABackbone=LLaMA3-8B, # Samples=88.90k2025.03 | 79.2 | |
| GPTAugBackbone=DeepSeekMath-7B, # Samples=88.62k2025.03 | 77.8 | |
| RefAug-90kBackbone=LLaMA3-8B, # Samples=89.92k2025.03 | 77.4 | |
| RefAug-90kBackbone=DeepSeekMath-7B, # Samples=89.92k2025.03 | 76.7 | |
| RefAugBackbone=LLaMA3-8B, # Samples=29.94k2025.03 | 75.9 | |
| RefAugBackbone=DeepSeekMath-7B, # Samples=29.94k2025.03 | 75.5 | |
| RFTBackbone=DeepSeekMath-7B, # Samples=86.52k2025.03 | 73.3 | |
| GPTAugBackbone=LLaMA3-8B, # Samples=88.62k2025.03 | 72.1 | |
| ISCBackbone=LLaMA3-8B, # Samples=86.78k2025.03 | 70.8 | |
| SFTBackbone=DeepSeekMath-7B, # Samples=14.97k2025.03 | 68.1 | |
| RFTBackbone=LLaMA3-8B, # Samples=86.52k2025.03 | 67.3 | |
| ISCBackbone=DeepSeekMath-7B, # Samples=86.78k2025.03 | 66.3 | |
| SFTBackbone=LLaMA3-8B, # Samples=14.97k2025.03 | 65.5 |