Mathematical Reasoning on Math Evaluation Suite
28.1Math ScoreSparse upcycling
Evaluation results
| Method | Links | |
|---|---|---|
| Sparse upcyclingTraining Context=Data-Matching (DM), Active Experts (Top-k)=22024.03 | 28.1 | |
| LLEMMAModel Size=7B2024.03 | 28 | |
| BTXActive Experts (Top-k)=22024.03 | 27.4 | |
| BTXActive Experts (Top-k)=1, Sampling Strategy=Sample2024.03 | 26.4 | |
| BTMActive Experts (Top-k)=22024.03 | 21.5 | |
| BTMActive Experts (Top-k)=12024.03 | 21.3 | |
| DenseTraining Context=Data-Matching (DM)2024.03 | 18.3 | |
| LLAMA-2Model Size=13B2024.03 | 16.3 | |
| LLAMA-2Model Size=7B2024.03 | 8.6 | |
| CODELLAMAModel Size=7B2024.03 | 8.1 |