Multi-step Reasoning (Math & Code) on GSM8K, MBPP, and HumanEval
49GSM8K AccuracyRED
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| REDParadigm=EDistill, Size=4.0B, Teacher=Qwen2.5-7B, # Tokens=18B, Dataset=Mixed-2.02026.05 | 49 | 28 | 23 | 33 | |
| REDParadigm=EDistill, Model Size=1.5B, Teacher=Llama3.2-3B, # Tokens=10B, Dataset=Mixed-1.12026.05 | 44 | 21 | 19 | 28 | |
| REDParadigm=EDistill, Model Size=1.7B, Teacher=Qwen2.5-3B, # Tokens=20B, Dataset=Mixed-1.12026.05 | 42 | 16 | 16 | 25 | |
| MiniCPMParadigm=Compute-Intensive Full-parameter, Model Size=1.0B, Teacher=-, # Tokens=1T, Dataset=N/A2026.05 | 39 | 32 | 6 | 26 | |
| Gemma3Paradigm=Full-Parameter, Size=4.0B, Teacher=–, # Tokens=4T, Dataset=N/A2026.05 | 38 | 47 | 35 | 40 | |
| LRCParadigm=EDistill, Size=4.0B, Teacher=Qwen2.5-7B, # Tokens=18B, Dataset=Mixed-2.02026.05 | 34 | 14 | 7 | 18 | |
| SmolLM2Paradigm=Compute-Intensive Full-parameter, Model Size=1.7B, Teacher=-, # Tokens=11T, Dataset=SmolLM2026.05 | 30 | 34 | 1 | 22 | |
| Llama3.2Paradigm=Full-Parameter, Size=3.0B, Teacher=Llama3.1-8/70B, # Tokens=9T, Dataset=N/A2026.05 | 25 | 38 | 27 | 30 | |
| MinitronParadigm=Full-Parameter, Size=4.0B, Teacher=Minitron-15B, # Tokens=94B, Dataset=N/A2026.05 | 25 | 35 | 4 | 21 | |
| InternLM2Paradigm=Compute-Intensive Full-parameter, Model Size=1.8B, Teacher=-, # Tokens=2T, Dataset=N/A2026.05 | 23 | 25 | 1 | 16 | |
| LLMStr.Paradigm=EDistill, Size=5.4B, Teacher=Llama3.1-8B, # Tokens=1.3B, Dataset=SlimPajama2026.05 | 22 | 23 | 16 | 20 | |
| LRCParadigm=EDistill, Model Size=1.5B, Teacher=Llama3.2-3B, # Tokens=10B, Dataset=Mixed-1.12026.05 | 21 | 7 | 0 | 9 | |
| REDParadigm=EDistill, Model Size=2.7B, Teacher=Llama2-7B, # Tokens=10B, Dataset=RedPajama2026.05 | 15 | 14 | 1 | 10 | |
| LRCParadigm=EDistill, Model Size=1.7B, Teacher=Qwen2.5-3B, # Tokens=20B, Dataset=Mixed-1.12026.05 | 9 | 9 | 1 | 6 | |
| LRCParadigm=EDistill, Model Size=2.7B, Teacher=Llama2-7B, # Tokens=10B, Dataset=RedPajama2026.05 | 9 | 7 | 1 | 6 | |
| Llama3.2Paradigm=Compute-Intensive Full-parameter, Model Size=1.2B, Teacher=Llama3.1-8B, # Tokens=9T, Dataset=N/A2026.05 | 7 | 27 | 18 | 17 | |
| SliceGPTParadigm=LoRA-style, Size=4.7B, Teacher=Llama2-7B, # Tokens=50M, Dataset=Alpaca2026.05 | 3 | 4 | 1 | 3 | |
| LLMStr.Paradigm=EDistill, Size=4.7B, Teacher=Llama2-7B, # Tokens=60M, Dataset=SlimPajama2026.05 | 3 | 11 | 0 | 5 | |
| Gemma3Paradigm=Compute-Intensive Full-parameter, Model Size=1.0B, Teacher=-, # Tokens=2T, Dataset=N/A2026.05 | 3 | 9 | 6 | 6 | |
| LLM-PrunerParadigm=LoRA-style, Size=4.7B, Teacher=Llama2-7B, # Tokens=50M, Dataset=Alpaca2026.05 | 1 | 2 | 0 | 1 |