Mathematical Reasoning on Mathematics (Pass@1)
65.8Pass@1LEMMA (w/ MetaMath)
Evaluation results
| Method | Links | |
|---|---|---|
| LEMMA (w/ MetaMath)Backbone=DeepSeekMath-7B, # Samples=403.59k2025.03 | 65.8 | |
| LEMMABackbone=DeepSeekMath-7B, # Samples=88.90k2025.03 | 61.6 | |
| RefAug-90kBackbone=DeepSeekMath-7B, # Samples=89.92k2025.03 | 57.5 | |
| RefAugBackbone=DeepSeekMath-7B, # Samples=29.94k2025.03 | 56.9 | |
| GPTAugBackbone=DeepSeekMath-7B, # Samples=88.62k2025.03 | 52.6 | |
| MetaMathBackbone=DeepSeekMath-7B, # Samples=394.99k2025.03 | 49 | |
| RFTBackbone=DeepSeekMath-7B, # Samples=86.52k2025.03 | 46.2 | |
| LEMMA (w/ MetaMath)Backbone=LLaMA3-8B, # Samples=403.59k2025.03 | 45.8 | |
| ISCBackbone=DeepSeekMath-7B, # Samples=86.78k2025.03 | 43.1 | |
| SFTBackbone=DeepSeekMath-7B, # Samples=14.97k2025.03 | 39.6 | |
| LEMMABackbone=LLaMA3-8B, # Samples=88.90k2025.03 | 39.2 | |
| GPTAugBackbone=LLaMA3-8B, # Samples=88.62k2025.03 | 36.5 | |
| RefAug-90kBackbone=LLaMA3-8B, # Samples=89.92k2025.03 | 35.7 | |
| RefAugBackbone=LLaMA3-8B, # Samples=29.94k2025.03 | 35.5 | |
| MetaMathBackbone=LLaMA3-8B, # Samples=394.99k2025.03 | 35.3 | |
| ISCBackbone=LLaMA3-8B, # Samples=86.78k2025.03 | 31.8 | |
| RFTBackbone=LLaMA3-8B, # Samples=86.52k2025.03 | 24.9 | |
| SFTBackbone=LLaMA3-8B, # Samples=14.97k2025.03 | 23.5 |