Mathematical Reasoning on GSM8K (EM)
97.04EMPhi-4-mini + Mistral3-3B
Evaluation results
| Method | Links | |
|---|---|---|
| Phi-4-mini + Mistral3-3BTraining=CORE2026.01 | 97.04 | |
| SafeThinkerModel=Qwen2.5-7B-Instruct2026.01 | 89.8 | |
| Tulu 3 CodeSFT Dataset=Tulu 3 Code, Base Model=DeepSeek-R1-Distill-Qwen-32B2024.10 | 89.2 | |
| SafeDecodingModel=Qwen2.5-7B-Instruct2026.01 | 89.1 | |
| MATRIX-Gen-ReasonSFT Dataset=MATRIX-Gen-Reason, Base Model=DeepSeek-R1-Distill-Qwen-32B, Samples=10k2024.10 | 88.7 | |
| No DefenseModel=Qwen2.5-7B-Instruct2026.01 | 88.7 | |
| Self-ReminderModel=Qwen2.5-7B-Instruct2026.01 | 88.7 | |
| Camel MATHSFT Dataset=Camel MATH, Base Model=DeepSeek-R1-Distill-Qwen-32B2024.10 | 88.6 | |
| Numina MATHSFT Dataset=Numina MATH, Base Model=DeepSeek-R1-Distill-Qwen-32B2024.10 | 88.5 | |
| Self-ExaminationModel=Qwen2.5-7B-Instruct2026.01 | 88.5 | |
| PPLModel=Qwen2.5-7B-Instruct2026.01 | 87.8 | |
| Magpie ReasonSFT Dataset=Magpie Reason, Base Model=DeepSeek-R1-Distill-Qwen-32B2024.10 | 87.7 | |
| Tulu 3 MATHSFT Dataset=Tulu 3 MATH, Base Model=DeepSeek-R1-Distill-Qwen-32B2024.10 | 87.4 | |
| PersonaHub-MathSFT Dataset=PersonaHub-Math, Base Model=DeepSeek-R1-Distill-Qwen-32B2024.10 | 87.3 | |
| Phi-4-mini + Mistral3-3B + OracleTraining=SD-E²2026.01 | 86.63 | |
| CoIPOModel=Qwen2.5-72B2026.02 | 86.28 | |
| Qwen2.5-7B-InstructTraining=SD-E²2026.01 | 86.21 | |
| CoIPOModel=Qwen2.5-14B2026.02 | 84.76 | |
| Ministral-3-8B-ReasoningTraining=SD-E²2026.01 | 84.5 | |
| ICDModel=Qwen2.5-7B-Instruct2026.01 | 83.2 | |
| Trajectory ReplayDonor=QWEN3-VL-8B, Target=GLM-4.6V, Type=DENSE->REAS2026.01 | 81.5 | |
| BaseDonor=QWEN3-VL-8B, Target=GLM-4.6V, Type=DENSE->REAS2026.01 | 81.2 | |
| BaseModel=Qwen2.5-72B2026.02 | 80.97 | |
| BaseDonor=QWEN3-VL-2B, Target=MINISTRAL-3-14B, Type=CROSS-FAM2026.01 | 78.1 | |
| BaseDonor=MINISTRAL-3-3B, Target=MINISTRAL-3-14B, Type=IN-FAM2026.01 | 78.1 | |
| Phi-3-small-8k-InstructTraining=SD-E²2026.01 | 78 | |
| Trajectory ReplayDonor=MINISTRAL-3-3B, Target=MINISTRAL-3-14B, Type=IN-FAM2026.01 | 78 | |
| Trajectory ReplayDonor=QWEN3-VL-2B, Target=MINISTRAL-3-14B, Type=CROSS-FAM2026.01 | 77.9 | |
| BaseModel=Qwen2.5-14B2026.02 | 75.66 | |
| CoIPOModel=Qwen2.5-7B2026.02 | 74.37 | |
| No DefenseModel=Llama-3-8B-Instruct2026.01 | 72.3 | |
| SafeDecodingModel=Llama-3-8B-Instruct2026.01 | 72.2 | |
| SafeThinkerModel=Llama-3-8B-Instruct2026.01 | 72.2 | |
| BaseDonor=MINISTRAL-3-3B, Target=QWEN3-VL-8B, Type=REVERSE2026.01 | 72 | |
| BaseDonor=QWEN3-VL-2B, Target=QWEN3-VL-8B, Type=IN-FAM2026.01 | 72 | |
| Trajectory ReplayDonor=QWEN3-VL-2B, Target=QWEN3-VL-8B, Type=IN-FAM2026.01 | 71.9 | |
| Trajectory ReplayDonor=MINISTRAL-3-3B, Target=QWEN3-VL-8B, Type=REVERSE2026.01 | 71.5 | |
| Self-ReminderModel=Llama-3-8B-Instruct2026.01 | 71 | |
| Self-ExaminationModel=Llama-3-8B-Instruct2026.01 | 70.8 | |
| BaseModel=Qwen2.5-7B2026.02 | 70.05 | |
| PPLModel=Llama-3-8B-Instruct2026.01 | 69.6 | |
| ICDModel=Llama-3-8B-Instruct2026.01 | 68.9 | |
| BaseDonor=QWEN3-VL-8B, Target=GPT-OSS-20B, Type=DENSE->MOE2026.01 | 68.5 | |
| Trajectory ReplayDonor=QWEN3-VL-8B, Target=GPT-OSS-20B, Type=DENSE->MOE2026.01 | 68.2 | |
| IPOBase model=Qwen-2 instruct 7B2025.05 | 64.06 | |
| SamPOBase model=Qwen-2 instruct 7B2025.05 | 61.33 | |
| SGDPOBase model=Qwen-2 instruct 7B2025.05 | 61.11 | |
| DPOBase model=Qwen-2 instruct 7B2025.05 | 59.89 | |
| CoT-Influx#Input CoT shots=48, #Average tokens=2037, Backbone=LLaMA2-70B2023.12 | 59.59 | |
| TDPOBase model=Qwen-2 instruct 7B2025.05 | 59.51 | |
| NCABase model=Qwen-2 instruct 7B2025.05 | 59.21 | |
| BCOBase model=Qwen-2 instruct 7B2025.05 | 58.98 | |
| BaseDonor=QWEN3-VL-8B, Target=MINISTRAL-3-3B, Type=CROSS-FAM2026.01 | 56.4 | |
| SFTBase model=Qwen-2 instruct 7B2025.05 | 56.25 | |
| Few-shot-CoT#Input CoT shots=8, #Average tokens=655, Backbone=LLaMA2-70B2023.12 | 55.42 | |
| Trajectory ReplayDonor=QWEN3-VL-8B, Target=MINISTRAL-3-3B, Type=CROSS-FAM2026.01 | 55.2 | |
| TopK retrieval#Input CoT shots=20, #Average tokens=3535.4, Backbone=LLaMA2-70B2023.12 | 54.59 | |
| BM25 retrieval#Input CoT shots=20, #Average tokens=3816.1, Backbone=LLaMA2-70B2023.12 | 54.21 | |
| 27B w/ mHC# Shots=8-shot2025.12 | 53.8 | |
| 27B w/ HC# Shots=8-shot2025.12 | 53.2 | |
| Random retrieval#Input CoT shots=20, #Average tokens=3379.8, Backbone=LLaMA2-70B2023.12 | 53.07 | |
| 27B Baseline# Shots=8-shot2025.12 | 46.7 | |
| BaseDonor=MINISTRAL-3-14B, Target=QWEN3-VL-2B, Type=REVERSE2026.01 | 44.2 | |
| Trajectory ReplayDonor=MINISTRAL-3-14B, Target=QWEN3-VL-2B, Type=REVERSE2026.01 | 43.8 | |
| 13B (Multi-Task Tuning)Tuning=Multi-task2024.06 | 39.03 | |
| Latent Thinking Optimization w. LRMModel=Huginn-3.5B2025.09 | 38.5 | |
| Weighted Majority Voting w. LRMModel=Huginn-3.5B2025.09 | 37.5 | |
| 7B-Expert (GSM8K)Tuning=Single-task2024.06 | 37.07 | |
| Proxy TuningContext=Multi-task transfer from 7B2024.06 | 34.87 | |
| 7B (Multi-Task Tuning)Context=Multi-task transfer from 7B2024.06 | 34.72 | |
| Self-Correction w. Confidence ScoreModel=Huginn-3.5B2025.09 | 34.2 | |
| Majority VotingModel=Huginn-3.5B2025.09 | 33.3 | |
| Latent Thinking Correction w. CoE-RModel=Huginn-3.5B2025.09 | 33 | |
| Base ModelModel=Huginn-3.5B2025.09 | 32.6 | |
| Latent Thinking Correction w. CoE-CModel=Huginn-3.5B2025.09 | 32.4 | |
| CoT-Influx#Input CoT shots=48, #Average tokens=2037, Backbone=LLaMA2-13B2023.12 | 32.37 | |
| Few-shot-CoT#Input CoT shots=8, #Average tokens=655, Backbone=LLaMA2-13B2023.12 | 27.82 | |
| Self-Correction w. Verbal EvaluationModel=Huginn-3.5B2025.09 | 26.2 | |
| BM25 retrieval#Input CoT shots=20, #Average tokens=3816.1, Backbone=LLaMA2-13B2023.12 | 25.17 | |
| TopK+GPT4 Compression#Input CoT shots=40, #Average tokens=1376, Backbone=LLaMA2-70B2023.12 | 25.17 | |
| TopK retrieval#Input CoT shots=20, #Average tokens=3535.4, Backbone=LLaMA2-13B2023.12 | 23.65 | |
| TopK+LLMLingua#Input CoT shots=40, #Average tokens=2048, Backbone=LLaMA2-70B2023.12 | 22.74 | |
| Random retrieval#Input CoT shots=20, #Average tokens=3379.8, Backbone=LLaMA2-13B2023.12 | 22.21 | |
| Zero-shot-CoT#Input CoT shots=0, Backbone=LLaMA2-70B2023.12 | 21.91 | |
| Proxy TuningContext=Multi-task transfer from 1.1B2024.06 | 18.65 | |
| CoT-Influx#Input CoT shots=48, #Average tokens=2037, Backbone=LLaMA2-7B2023.12 | 15.92 | |
| TopK retrieval#Input CoT shots=20, #Average tokens=3535.4, Backbone=LLaMA2-7B2023.12 | 14.56 | |
| 1.1B (Multi-Task Tuning)Context=Multi-task transfer from 1.1B2024.06 | 14.4 | |
| Few-shot-CoT#Input CoT shots=8, #Average tokens=655, Backbone=LLaMA2-7B2023.12 | 13.79 | |
| LLAMA2-7Bshot=8-shot2023.10 | 13.7 | |
| BM25 retrieval#Input CoT shots=20, #Average tokens=3816.1, Backbone=LLaMA2-7B2023.12 | 13.42 | |
| Random retrieval#Input CoT shots=20, #Average tokens=3379.8, Backbone=LLaMA2-7B2023.12 | 12.51 | |
| 1.1B-Expert (GSM8K)Tuning=Single-task2024.06 | 12.51 | |
| Zero-shot-CoT#Input CoT shots=0, Backbone=LLaMA2-13B2023.12 | 12.28 | |
| Zero-shot#Input CoT shots=0, Backbone=LLaMA2-70B2023.12 | 11.45 | |
| TopK+GPT4 Compression#Input CoT shots=40, #Average tokens=1376, Backbone=LLaMA2-13B2023.12 | 11.01 | |
| TopK+LLMLingua#Input CoT shots=40, #Average tokens=2048, Backbone=LLaMA2-13B2023.12 | 8.34 | |
| 7B-Expert (TruthfulQA)Tuning=Single-task2024.06 | 8.26 | |
| BaseModel=Llama-7B2026.02 | 7.2 | |
| TopK+GPT4 Compression#Input CoT shots=40, #Average tokens=1376, Backbone=LLaMA2-7B2023.12 | 7.08 |