Code Generation on HumanE
85.2AccuracyDenser
Evaluation Results
| Method | Links | |
|---|---|---|
| DenserBackbone=Qwen3-32B-think2025.12 | 85.2 | |
| Process SupervisionBackbone=Qwen3-32B-think2025.12 | 84.8 | |
| Reflection-CoTBackbone=Qwen3-32B-think2025.12 | 84.2 | |
| DenserBackbone=Qwen3-32B2025.12 | 83.6 | |
| Self-VerificationBackbone=Qwen3-32B-think2025.12 | 83.1 | |
| Tree-of-ThoughtBackbone=Qwen3-32B-think2025.12 | 83 | |
| Process SupervisionBackbone=Qwen3-32B2025.12 | 82.7 | |
| Self-ConsistencyBackbone=Qwen3-32B-think2025.12 | 82.4 | |
| Reflection-CoTBackbone=Qwen3-32B2025.12 | 82.1 | |
| Think-to-ThinkBackbone=Qwen3-32B-think2025.12 | 81.7 | |
| Self-VerificationBackbone=Qwen3-32B2025.12 | 81.4 | |
| Tree-of-ThoughtBackbone=Qwen3-32B2025.12 | 80.9 | |
| Self-ConsistencyBackbone=Qwen3-32B2025.12 | 80.3 | |
| Chain-of-ThoughtBackbone=Qwen3-32B-think2025.12 | 79.7 | |
| Think-to-ThinkBackbone=Qwen3-32B2025.12 | 79.6 | |
| DenserBackbone=Qwen3-8B-think2025.12 | 79.1 | |
| Process SupervisionBackbone=Qwen3-8B-think2025.12 | 78.5 | |
| Reflection-CoTBackbone=Qwen3-8B-think2025.12 | 77.9 | |
| Chain-of-ThoughtBackbone=Qwen3-32B2025.12 | 77.8 | |
| DenserBackbone=Qwen3-8B2025.12 | 77.3 | |
| Self-VerificationBackbone=Qwen3-8B-think2025.12 | 76.9 | |
| Tree-of-ThoughtBackbone=Qwen3-8B-think2025.12 | 76.8 | |
| DenserBackbone=Qwen3-14B-no-think2025.12 | 76.8 | |
| Process SupervisionBackbone=Qwen3-8B2025.12 | 76.3 | |
| Self-ConsistencyBackbone=Qwen3-8B-think2025.12 | 76.2 | |
| DenserBackbone=Qwen3-4B-think2025.12 | 76.1 | |
| Process SupervisionBackbone=Qwen3-14B-no-think2025.12 | 76 | |
| Reflection-CoTBackbone=Qwen3-8B2025.12 | 75.7 | |
| EMoEBase Model=ERNIE-4.5-21B-A3B, Trained Activated Experts=6+2, MoE Routing Method=EMoE, Inference Budget (k')=12 + 22025.09 | 75.61 | |
| Think-to-ThinkBackbone=Qwen3-8B-think2025.12 | 75.5 | |
| Reflection-CoTBackbone=Qwen3-14B-no-think2025.12 | 75.4 | |
| Process SupervisionBackbone=Qwen3-4B-think2025.12 | 75.2 | |
| Self-VerificationBackbone=Qwen3-8B2025.12 | 74.8 | |
| Reflection-CoTBackbone=Qwen3-4B-think2025.12 | 74.6 | |
| Self-VerificationBackbone=Qwen3-14B-no-think2025.12 | 74.6 | |
| Tree-of-ThoughtBackbone=Qwen3-8B2025.12 | 74.5 | |
| Tree-of-ThoughtBackbone=Qwen3-14B-no-think2025.12 | 74.2 | |
| Self-ConsistencyBackbone=Qwen3-8B2025.12 | 73.9 | |
| Chain-of-ThoughtBackbone=Qwen3-8B-think2025.12 | 73.8 | |
| Self-VerificationBackbone=Qwen3-4B-think2025.12 | 73.7 | |
| DenserBackbone=Qwen3-4B2025.12 | 73.6 | |
| Self-ConsistencyBackbone=Qwen3-14B-no-think2025.12 | 73.6 | |
| Tree-of-ThoughtBackbone=Qwen3-4B-think2025.12 | 73.4 | |
| Think-to-ThinkBackbone=Qwen3-8B2025.12 | 73.2 | |
| EMoEBase Model=ERNIE-4.5-21B-A3B, Trained Activated Experts=6+2, MoE Routing Method=EMoE, Inference Budget (k')=6 + 22025.09 | 73.17 | |
| Think-to-ThinkBackbone=Qwen3-14B-no-think2025.12 | 72.9 | |
| Self-ConsistencyBackbone=Qwen3-4B-think2025.12 | 72.8 | |
| Process SupervisionBackbone=Qwen3-4B2025.12 | 72.7 | |
| Reflection-CoTBackbone=Qwen3-4B2025.12 | 72.1 | |
| Think-to-ThinkBackbone=Qwen3-4B-think2025.12 | 72.1 | |
| Chain-of-ThoughtBackbone=Qwen3-8B2025.12 | 71.5 | |
| Top-kBase Model=ERNIE-4.5-21B-A3B, Trained Activated Experts=6+2, MoE Routing Method=Top-k, Inference Budget (k')=6 + 22025.09 | 71.34 | |
| Self-VerificationBackbone=Qwen3-4B2025.12 | 71.3 | |
| Chain-of-ThoughtBackbone=Qwen3-14B-no-think2025.12 | 71.2 | |
| Tree-of-ThoughtBackbone=Qwen3-4B2025.12 | 71 | |
| EMoEBase Model=ERNIE-4.5-21B-A3B, Trained Activated Experts=6+2, MoE Routing Method=EMoE, Inference Budget (k')=3 + 22025.09 | 70.73 | |
| Self-ConsistencyBackbone=Qwen3-4B2025.12 | 70.5 | |
| Chain-of-ThoughtBackbone=Qwen3-4B-think2025.12 | 70.4 | |
| Think-to-ThinkBackbone=Qwen3-4B2025.12 | 69.7 | |
| Top-kBase Model=ERNIE-4.5-21B-A3B, Trained Activated Experts=6+2, MoE Routing Method=Top-k, Inference Budget (k')=12 + 22025.09 | 68.29 | |
| Chain-of-ThoughtBackbone=Qwen3-4B2025.12 | 68.2 | |
| Top-kBase Model=ERNIE-4.5-21B-A3B, Trained Activated Experts=6+2, MoE Routing Method=Top-k, Inference Budget (k')=3 + 22025.09 | 66.46 | |
| EMoEBase Model=DeepSeek-V2-Lite, Trained Activated Experts=6+2, MoE Routing Method=EMoE, Inference Budget (k')=12 + 22025.09 | 41.46 | |
| EMoEBase Model=DeepSeek-V2-Lite, Trained Activated Experts=6+2, MoE Routing Method=EMoE, Inference Budget (k')=6 + 22025.09 | 39.63 | |
| Top-kBase Model=DeepSeek-V2-Lite, Trained Activated Experts=6+2, MoE Routing Method=Top-k, Inference Budget (k')=12 + 22025.09 | 39.02 | |
| Top-kBase Model=DeepSeek-V2-Lite, Trained Activated Experts=6+2, MoE Routing Method=Top-k, Inference Budget (k')=6 + 22025.09 | 36.59 | |
| EMoEBase Model=DeepSeek-V2-Lite, Trained Activated Experts=6+2, MoE Routing Method=EMoE, Inference Budget (k')=3 + 22025.09 | 34.76 | |
| Top-kBase Model=DeepSeek-V2-Lite, Trained Activated Experts=6+2, MoE Routing Method=Top-k, Inference Budget (k')=3 + 22025.09 | 28.66 | |
| EMoEBase Model=OLMoE-1B-7B-0924, Trained Activated Experts=8, MoE Routing Method=EMoE, Inference Budget (k')=162025.09 | 21.95 | |
| Top-kBase Model=OLMoE-1B-7B-0924, Trained Activated Experts=8, MoE Routing Method=Top-k, Inference Budget (k')=82025.09 | 21.34 | |
| EMoEBase Model=LoRAMoE, Trained Activated Experts=2, MoE Routing Method=EMoE, Inference Budget (k')=62025.09 | 20.73 | |
| Top-kBase Model=LoRAMoE, Trained Activated Experts=2, MoE Routing Method=Top-k, Inference Budget (k')=22025.09 | 20.24 | |
| EMoEBase Model=OLMoE-1B-7B-0924, Trained Activated Experts=8, MoE Routing Method=EMoE, Inference Budget (k')=82025.09 | 20.12 | |
| Top-kBase Model=LoRAMoE, Trained Activated Experts=2, MoE Routing Method=Top-k, Inference Budget (k')=42025.09 | 19.39 | |
| EMoEBase Model=LoRAMoE, Trained Activated Experts=2, MoE Routing Method=EMoE, Inference Budget (k')=22025.09 | 18.9 | |
| Top-kBase Model=LoRAMoE, Trained Activated Experts=2, MoE Routing Method=Top-k, Inference Budget (k')=62025.09 | 18.7 | |
| EMoEBase Model=LoRAMoE, Trained Activated Experts=2, MoE Routing Method=EMoE, Inference Budget (k')=42025.09 | 18.29 | |
| EMoEBase Model=LoRAMoE, Trained Activated Experts=2, MoE Routing Method=EMoE, Inference Budget (k')=12025.09 | 17.68 | |
| Top-kBase Model=OLMoE-1B-7B-0924, Trained Activated Experts=8, MoE Routing Method=Top-k, Inference Budget (k')=162025.09 | 17.07 | |
| Top-kBase Model=LoRAMoE, Trained Activated Experts=2, MoE Routing Method=Top-k, Inference Budget (k')=12025.09 | 15.97 | |
| EMoEBase Model=OLMoE-1B-7B-0924, Trained Activated Experts=8, MoE Routing Method=EMoE, Inference Budget (k')=42025.09 | 15.85 | |
| Top-kBase Model=OLMoE-1B-7B-0924, Trained Activated Experts=8, MoE Routing Method=Top-k, Inference Budget (k')=42025.09 | 13.41 |