Mathematical Reasoning on MATHEMATICS (Accuracy)
74.1AccuracyLEMMA
Evaluation Results
| Method | Links | |
|---|---|---|
| LEMMABackbone=Qwen2-Math-7B, # Samples=88.90k2025.03 | 74.1 | |
| RefAug-90kBackbone=Qwen2-Math-7B, # Samples=89.92k2025.03 | 68.7 | |
| GPTAugBackbone=Qwen2-Math-7B, # Samples=88.62k2025.03 | 64.9 | |
| RefAugBackbone=Qwen2-Math-7B, # Samples=29.94k2025.03 | 62.7 | |
| S2LTarget Model=Phi-3-Mini (3.8B), Fine-tuning Data=50%, Evaluation Protocol=Zero-shot2024.03 | 62.1 | |
| MetaMathBackbone=Qwen2-Math-7B, # Samples=394.99k2025.03 | 60.7 | |
| Full TrainingTarget Model=Phi-3-Mini (3.8B), Fine-tuning Data=100%, Evaluation Protocol=Zero-shot2024.03 | 60.2 | |
| RFTBackbone=Qwen2-Math-7B, # Samples=86.52k2025.03 | 57.4 | |
| ISCBackbone=Qwen2-Math-7B, # Samples=86.78k2025.03 | 51.9 | |
| SFTBackbone=Qwen2-Math-7B, # Samples=14.97k2025.03 | 50.3 | |
| S2LTarget Model=Llama-2-7B, Fine-tuning Data=50%, Evaluation Protocol=Zero-shot2024.03 | 45.2 | |
| PretrainedTarget Model=Phi-3-Mini (3.8B), Evaluation Protocol=Zero-shot2024.03 | 44.3 | |
| Full TrainingTarget Model=Llama-2-7B, Fine-tuning Data=262K, Evaluation Protocol=Zero-shot2024.03 | 43.9 | |
| LEMMABackbone=Mistral-7B-v0.1, # Samples=88.90k2025.03 | 40.3 | |
| GREATSSelection Protocol=batch-wise2026.04 | 38.19 | |
| GradNormSelection Protocol=source-wise2026.04 | 38 | |
| COLMSelection Protocol=source-wise2026.04 | 38 | |
| COLMSelection Protocol=batch-wise2026.04 | 37.4 | |
| UniPROTSelection Protocol=batch-wise2026.04 | 37.2 | |
| UniPROTSelection Protocol=source-wise2026.04 | 36.9 | |
| GradNormSelection Protocol=batch-wise2026.04 | 36.5 | |
| GREATSSelection Protocol=source-wise2026.04 | 35.6 | |
| RefAug-90kBackbone=Mistral-7B-v0.1, # Samples=89.92k2025.03 | 34.8 | |
| GPTAugBackbone=Mistral-7B-v0.1, # Samples=88.62k2025.03 | 34.6 | |
| SBERTSelection Protocol=source-wise2026.04 | 34.05 | |
| SBERTSelection Protocol=batch-wise2026.04 | 34 | |
| RefAugBackbone=Mistral-7B-v0.1, # Samples=29.94k2025.03 | 33.7 | |
| FTBase Model=PHI-32026.04 | 33.3 | |
| MaxLossSelection Protocol=source-wise2026.04 | 31.45 | |
| MaxLossSelection Protocol=batch-wise2026.04 | 30.06 | |
| ISCBackbone=Mistral-7B-v0.1, # Samples=86.78k2025.03 | 27.4 | |
| MetaMathBackbone=Mistral-7B-v0.1, # Samples=394.99k2025.03 | 26.6 | |
| GREATSSelection Strategy=source-wise2026.04 | 21.1 | |
| UniPROTSelection Strategy=source-wise2026.04 | 20.9 | |
| GREATSSelection Strategy=batch-wise2026.04 | 20.7 | |
| GradNormSelection Strategy=batch-wise2026.04 | 20.4 | |
| GradNormSelection Strategy=source-wise2026.04 | 20.2 | |
| COLMSelection Strategy=batch-wise2026.04 | 19.8 | |
| UniPROTSelection Strategy=batch-wise2026.04 | 19.8 | |
| FT2026.04 | 19.4 | |
| COLMSelection Strategy=source-wise2026.04 | 18.9 | |
| SBERTSelection Strategy=source-wise2026.04 | 18.1 | |
| SFTBackbone=Mistral-7B-v0.1, # Samples=14.97k2025.03 | 16.6 | |
| RFTBackbone=Mistral-7B-v0.1, # Samples=86.52k2025.03 | 16.6 | |
| SBERTSelection Strategy=batch-wise2026.04 | 16.6 | |
| PretrainedTarget Model=Llama-2-7B, Evaluation Protocol=Zero-shot2024.03 | 8.3 |