Mathematical Reasoning on GSM (test)
65.4AccuracyTeacher
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| TeacherTeacher and Student Model Pair=Qwen-2.5-3b to Qwen-2.5-0.5b2025.10 | 65.4 | — | — | |
| TeacherTeacher and Student Model Pair=Llama-3.1-3b to Llama-3.1-1b2025.10 | 59.6 | — | — | |
| TeacherTeacher and Student Model Pair=Gemma-7b to Gemma-2b2025.10 | 36.9 | — | — | |
| AdaSwitchTeacher and Student Model Pair=Qwen-2.5-3b to Qwen-2.5-0.5b2025.10 | 36.3 | — | — | |
| SWITCHTeacher and Student Model Pair=Qwen-2.5-3b to Qwen-2.5-0.5b2025.10 | 35.2 | — | — | |
| AdaSwitchTeacher and Student Model Pair=Llama-3.1-3b to Llama-3.1-1b2025.10 | 35 | — | — | |
| ImitKDTeacher and Student Model Pair=Qwen-2.5-3b to Qwen-2.5-0.5b2025.10 | 34.6 | — | — | |
| On-policyTeacher and Student Model Pair=Qwen-2.5-3b to Qwen-2.5-0.5b2025.10 | 34.1 | — | — | |
| Off-policyTeacher and Student Model Pair=Qwen-2.5-3b to Qwen-2.5-0.5b2025.10 | 33.5 | — | — | |
| SWITCHTeacher and Student Model Pair=Llama-3.1-3b to Llama-3.1-1b2025.10 | 33.2 | — | — | |
| SKDTeacher and Student Model Pair=Qwen-2.5-3b to Qwen-2.5-0.5b2025.10 | 33 | — | — | |
| On-policyTeacher and Student Model Pair=Llama-3.1-3b to Llama-3.1-1b2025.10 | 32.7 | — | — | |
| SKDTeacher and Student Model Pair=Llama-3.1-3b to Llama-3.1-1b2025.10 | 32.4 | — | — | |
| SFTTeacher and Student Model Pair=Qwen-2.5-3b to Qwen-2.5-0.5b2025.10 | 31 | — | — | |
| Off-policyTeacher and Student Model Pair=Llama-3.1-3b to Llama-3.1-1b2025.10 | 30.6 | — | — | |
| ImitKDTeacher and Student Model Pair=Llama-3.1-3b to Llama-3.1-1b2025.10 | 30.6 | — | — | |
| SeqKDTeacher and Student Model Pair=Qwen-2.5-3b to Qwen-2.5-0.5b2025.10 | 30.4 | — | — | |
| SFTTeacher and Student Model Pair=Llama-3.1-3b to Llama-3.1-1b2025.10 | 29 | — | — | |
| SeqKDTeacher and Student Model Pair=Llama-3.1-3b to Llama-3.1-1b2025.10 | 28.2 | — | — | |
| AdaSwitchTeacher and Student Model Pair=Gemma-7b to Gemma-2b2025.10 | 26.8 | — | — | |
| On-policyTeacher and Student Model Pair=Gemma-7b to Gemma-2b2025.10 | 26.2 | — | — | |
| SKDTeacher and Student Model Pair=Gemma-7b to Gemma-2b2025.10 | 26 | — | — | |
| ImitKDTeacher and Student Model Pair=Gemma-7b to Gemma-2b2025.10 | 25.3 | — | — | |
| SWITCHTeacher and Student Model Pair=Gemma-7b to Gemma-2b2025.10 | 24.8 | — | — | |
| Off-policyTeacher and Student Model Pair=Gemma-7b to Gemma-2b2025.10 | 22.6 | — | — | |
| SeqKDTeacher and Student Model Pair=Gemma-7b to Gemma-2b2025.10 | 21.8 | — | — | |
| SFTTeacher and Student Model Pair=Gemma-7b to Gemma-2b2025.10 | 18.9 | — | — | |
| Alpaca-GPT4 + NAIT (7-11)System ID=12, Capability Features=All, Data sampling ratio=10%2026.03 | 16.53 | 37.2 | 3.24 | |
| Alpaca-GPT4 + NAIT (GSM)System ID=08, Capability Features=Individual, Data sampling ratio=10%2026.03 | 16 | 37.7 | 4.65 | |
| LIMASystem ID=02, Fine-tuning protocol=Full Fine-tuning, Data sampling ratio=10%2026.03 | 15.76 | 36.17 | 0.39 | |
| Alpaca-GPT4 + NAIT (MMLU)System ID=07, Capability Features=Individual, Data sampling ratio=10%2026.03 | 15.68 | 37.18 | 3.2 | |
| Alpaca-GPT4 + NAIT (CodeX)System ID=09, Capability Features=Individual, Data sampling ratio=10%2026.03 | 15.53 | 37.06 | 2.88 | |
| Alpaca-GPT4 + SelectITSystem ID=05, Fine-tuning protocol=Full Fine-tuning, Data sampling ratio=10%2026.03 | 15.4 | 37.16 | 3.15 | |
| Alpaca-GPT4System ID=01, Fine-tuning protocol=Full Fine-tuning, Data sampling ratio=10%2026.03 | 14.63 | 36.03 | — | |
| Alpaca-GPT4 + Q2QSystem ID=04, Fine-tuning protocol=Full Fine-tuning, Data sampling ratio=10%2026.03 | 14.5 | 35.68 | -0.98 | |
| Alpaca-GPT4 + Random BaselineSystem ID=06, Fine-tuning protocol=Full Fine-tuning, Data sampling ratio=10%2026.03 | 14.13 | 35.69 | -0.94 | |
| Alpaca-GPT4 + NAIT (TydiQA)System ID=10, Capability Features=Individual, Data sampling ratio=10%2026.03 | 13.8 | 36.71 | 1.89 | |
| Alpaca-GPT4 + AlpaGasusSystem ID=03, Fine-tuning protocol=Full Fine-tuning, Data sampling ratio=10%2026.03 | 13.34 | 35.18 | -2.34 | |
| Alpaca-GPT4 + NAIT (BBH)System ID=11, Capability Features=Individual, Data sampling ratio=10%2026.03 | 13.34 | 36.47 | 1.23 | |
| StudentTeacher and Student Model Pair=Llama-3.1-3b to Llama-3.1-1b2025.10 | 0 | — | — | |
| StudentTeacher and Student Model Pair=Qwen-2.5-3b to Qwen-2.5-0.5b2025.10 | 0 | — | — | |
| StudentTeacher and Student Model Pair=Gemma-7b to Gemma-2b2025.10 | 0 | — | — |