Code Reasoning on HumanEval
95.73HumanEval ScoreDeepSeek-R1-Distill-Qwen-14B (Reasoning)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DeepSeek-R1-Distill-Qwen-14B (Reasoning)Model Family=Qwen2.5-14B2026.01 | 95.73 | — | — | |
| DeepSeek-R1-Distill-Qwen-32B (Reasoning)Model Family=Qwen2.5-32B2026.01 | 92.41 | — | — | |
| ReasonAnyModel Family=Qwen2.5-14B2026.01 | 92.32 | — | — | |
| TIESModel Family=Qwen2.5-32B2026.01 | 91.46 | — | — | |
| ReasoningBase Model=Llama-3.1-8B2026.01 | 89.57 | — | — | |
| LinearModel Family=Qwen2.5-14B2026.01 | 89.02 | — | — | |
| Task ArithmeticModel Family=Qwen2.5-32B2026.01 | 88.41 | — | — | |
| Qwen2.5-Coder-ScaleQuest# Samples (K)=156, Base Model=Qwen2.5-Coder-7B-Base2024.10 | 86.6 | 69.9 | — | |
| ReasonAnyModel Family=Qwen2.5-32B2026.01 | 86.59 | — | — | |
| Qwen2.5-32B-Instruct (Safety)Model Family=Qwen2.5-32B2026.01 | 84.31 | — | — | |
| TIESModel Family=Qwen2.5-14B2026.01 | 84.15 | — | — | |
| Qwen2.5-Coder-CFB-Aug# Samples (K)=156, Base Model=Qwen2.5-Coder-7B-Base2024.10 | 84.1 | 69.3 | — | |
| LinearModel Family=Qwen2.5-32B2026.01 | 82.32 | — | — | |
| DAREModel Family=Qwen2.5-32B2026.01 | 82.32 | — | — | |
| Sigma-MoE-TinyArchitecture=MoE, # Activated Params=0.5B, # Total Params=20B2025.12 | 79.9 | — | — | |
| Qwen2.5-Coder-CFB# Samples (K)=156, Base Model=Qwen2.5-Coder-7B-Base2024.10 | 79.3 | 64 | — | |
| Qwen2.5-14B-Instruct (Safety)Model Family=Qwen2.5-14B2026.01 | 78.8 | — | — | |
| Phi-3.5-MoEArchitecture=MoE, # Activated Params=6.6B, # Total Params=42B2025.12 | 75 | — | — | |
| FuseLLMModel Family=Qwen2.5-32B2026.01 | 74.39 | — | — | |
| DeepSeek-R1-Distill-Llama-8BArchitecture=Dense, # Activated Params=8B, # Total Params=8B2025.12 | 73.2 | — | — | |
| TakeoverModel Scale=1.5B, System Configuration=LLM-involved2026.04 | 73.2 | — | — | |
| S2TModel Scale=1.5B, System Configuration=LLM-involved2026.04 | 72.6 | — | — | |
| R2RModel Scale=1.5B, System Configuration=LLM-involved2026.04 | 72.2 | — | — | |
| Qwen3-1.7BArchitecture=Dense, # Activated Params=1.7B, # Total Params=1.7B2025.12 | 70.1 | — | — | |
| SpecRModel Scale=1.5B, System Configuration=LLM-involved2026.04 | 68 | — | — | |
| ReasonAnyMerging Protocol=ReasonAny2026.01 | 67.66 | — | — | |
| DAREModel Family=Qwen2.5-14B2026.01 | 64.63 | — | — | |
| R2RModel Scale=0.5B, System Configuration=LLM-involved2026.04 | 64.6 | — | — | |
| DeepSeek-R1-Distill-Qwen-7BArchitecture=Dense, # Activated Params=7B, # Total Params=7B2025.12 | 64 | — | — | |
| SpecRModel Scale=0.5B, System Configuration=LLM-involved2026.04 | 62.4 | — | — | |
| TakeoverModel Scale=0.5B, System Configuration=LLM-involved2026.04 | 62 | — | — | |
| LEDModel Family=Qwen2.5-32B2026.01 | 61.59 | — | — | |
| Task ArithmeticModel Family=Qwen2.5-14B2026.01 | 61.59 | — | — | |
| S2TModel Scale=0.5B, System Configuration=LLM-involved2026.04 | 61 | — | — | |
| LinearMerging Protocol=Linear2026.01 | 59.15 | — | — | |
| S2TLocalModel Scale=1.5B, System Configuration=SLM-only2026.04 | 56.9 | — | — | |
| DistilModel Scale=1.5B, System Configuration=SLM-only2026.04 | 48.1 | — | — | |
| TSD-KDModel Scale=1.5B, System Configuration=SLM-only2026.04 | 47.8 | — | — | |
| Mul-TModel Scale=1.5B, System Configuration=SLM-only2026.04 | 44.7 | — | — | |
| TaHModel Scale=1.5B, System Configuration=SLM-only2026.04 | 43.6 | — | — | |
| SafetyBase Model=Llama-3.1-8B2026.01 | 42.94 | — | — | |
| GreedyModel Scale=1.5B, System Configuration=SLM-only2026.04 | 42.7 | — | — | |
| LEDModel Family=Qwen2.5-14B2026.01 | 40.85 | — | — | |
| SampleModel Scale=1.5B, System Configuration=SLM-only2026.04 | 39.2 | — | — | |
| LEDMerging Protocol=LED2026.01 | 38.66 | — | — | |
| OursBackbone=Qwen1.5-MoE-A2.7B, Sparsity=50%2026.06 | 38.1 | — | — | |
| OursQBackbone=Qwen1.5-MoE-A2.7B, Sparsity=25%, Quantization=4-bit2026.06 | 36.6 | — | — | |
| Teacher (Qwen3-0.6B)Role=Teacher, Model=Qwen3-0.6B, Evaluation Protocol=generation-based evaluation2026.03 | 35.3 | — | — | |
| BaselineBackbone=Qwen1.5-MoE-A2.7B, Type=–2026.06 | 34.2 | — | — | |
| TIESMerging Protocol=TIES2026.01 | 34.15 | — | — | |
| C-PruneBackbone=Qwen1.5-MoE-A2.7B, Sparsity=20%2026.06 | 32.9 | — | — | |
| BaselineBackbone=DeepSeek-V2-Lite, Type=–2026.06 | 32.3 | — | — | |
| OursBackbone=DeepSeek-V2-Lite, Sparsity=50%2026.06 | 28.7 | — | — | |
| S2TLocalModel Scale=0.5B, System Configuration=SLM-only2026.04 | 26.7 | — | — | |
| OursQBackbone=DeepSeek-V2-Lite, Sparsity=25%, Quantization=4-bit2026.06 | 23.8 | — | — | |
| TSD-KDModel Scale=0.5B, System Configuration=SLM-only2026.04 | 23 | — | — | |
| DistilModel Scale=0.5B, System Configuration=SLM-only2026.04 | 22.9 | — | — | |
| TaHModel Scale=0.5B, System Configuration=SLM-only2026.04 | 20.3 | — | — | |
| Hybrid KDASequence Mixer=KDA, Attention Layers=7, Evaluation Protocol=generation-based evaluation2026.03 | 19.9 | — | — | |
| Mul-TModel Scale=0.5B, System Configuration=SLM-only2026.04 | 19.7 | — | — | |
| FuseLLMMerging Protocol=FuseLLM2026.01 | 19.51 | — | — | |
| FuseLLMModel Family=Qwen2.5-14B2026.01 | 19.51 | — | — | |
| GreedyModel Scale=0.5B, System Configuration=SLM-only2026.04 | 18.9 | — | — | |
| C-PruneBackbone=DeepSeek-V2-Lite, Sparsity=20%2026.06 | 18.9 | — | — | |
| SampleModel Scale=0.5B, System Configuration=SLM-only2026.04 | 16.4 | — | — | |
| Pure KDASequence Mixer=KDA, Attention Layers=0, Evaluation Protocol=generation-based evaluation2026.03 | 12.8 | — | — | |
| Hybrid MambaSequence Mixer=Mamba2, Attention Layers=7, Evaluation Protocol=generation-based evaluation2026.03 | 11.2 | — | — | |
| Pure MambaSequence Mixer=Mamba2, Attention Layers=0, Evaluation Protocol=generation-based evaluation2026.03 | 6.1 | — | — | |
| DAREMerging Protocol=DARE2026.01 | 3.66 | — | — | |
| Task ArithmeticMerging Protocol=Task Arithmetic2026.01 | 0.61 | — | — | |
| CE-GPPOBase Model=DS-R1-Distill-Qwen-1.5B, Training Algorithm=CE-GPPO, Beta1 Hyperparameter=0.5, Beta2 Hyperparameter=12025.09 | — | 76.5 | — | |
| CE-GPPOBase Model=DS-R1-Distill-Qwen-1.5B, Training Algorithm=CE-GPPO, Beta1 Hyperparameter=0.75, Beta2 Hyperparameter=12025.09 | — | 74.9 | — | |
| CE-GPPOBase Model=DS-R1-Distill-Qwen-7B, Training Algorithm=CE-GPPO, Beta1 Hyperparameter=0.5, Beta2 Hyperparameter=12025.09 | — | 91.9 | — | |
| CE-GPPOBase Model=DS-R1-Distill-Qwen-7B, Training Algorithm=CE-GPPO, Beta1 Hyperparameter=0.75, Beta2 Hyperparameter=12025.09 | — | 93 | — | |
| DAPOBase Model=DS-R1-Distill-Qwen-1.5B, Training Algorithm=DAPO2025.09 | — | 73.2 | — | |
| DAPOBase Model=DS-R1-Distill-Qwen-7B, Training Algorithm=DAPO2025.09 | — | 92.5 | — | |
| DiReCTBackbone=GPT-2-Medium (355M)2026.05 | — | — | 3.8 | |
| DiReCTBackbone=Llama-1.1B2026.05 | — | — | 11.2 | |
| DS-R1-Distill-Qwen-1.5BBase Model=DS-R1-Distill-Qwen-1.5B2025.09 | — | 70.4 | — | |
| DS-R1-Distill-Qwen-7BBase Model=DS-R1-Distill-Qwen-7B2025.09 | — | 89.6 | — | |
| GradNorm (IS)Backbone=GPT-2-Medium (355M)2026.05 | — | — | 1.8 | |
| GradNorm (IS)Backbone=Llama-1.1B2026.05 | — | — | 8.3 | |
| GRPOBase Model=DS-R1-Distill-Qwen-1.5B, Training Algorithm=GRPO2025.09 | — | 67.5 | — | |
| GRPOBase Model=DS-R1-Distill-Qwen-7B, Training Algorithm=GRPO2025.09 | — | 88.6 | — | |
| InfoBatchBackbone=GPT-2-Medium (355M)2026.05 | — | — | 3 | |
| InfoBatchBackbone=Llama-1.1B2026.05 | — | — | 7.1 | |
| Loss-basedBackbone=GPT-2-Medium (355M)2026.05 | — | — | 1.2 | |
| Loss-basedBackbone=Llama-1.1B2026.05 | — | — | 6.8 | |
| Perplexity-basedBackbone=GPT-2-Medium (355M)2026.05 | — | — | 1.8 | |
| Perplexity-basedBackbone=Llama-1.1B2026.05 | — | — | 7.6 | |
| Uniform SamplingBackbone=GPT-2-Medium (355M)2026.05 | — | — | 1.2 | |
| Uniform SamplingBackbone=Llama-1.1B2026.05 | — | — | 6.2 |