Code Generation on MBPP+ (Pass@1, AVG)
75.4Pass@1CODEBLOCK
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| CODEBLOCKBase model=Qwen2.5-Coder-7B-Instruct, Eff. Tokens (%)=1.92026.06 | 75.4 | — | |
| DS2Base model=Qwen2.5-Coder-7B-Instruct, Eff. Tokens (%)=4.62026.06 | 74.1 | — | |
| CLAMBase model=Qwen2.5-Coder-7B-Instruct, Eff. Tokens (%)=3.52026.06 | 74.1 | — | |
| TOKEN CLEANINGBase model=Qwen2.5-Coder-7B-Instruct, Eff. Tokens (%)=1.72026.06 | 73.3 | — | |
| RANDOM SELECTIONBase model=Qwen2.5-Coder-7B-Instruct, Eff. Tokens (%)=1.92026.06 | 73.1 | — | |
| MTModel=Qwen2.5-Coder-14B-Inst2026.04 | 72.8 | 84.07 | |
| DS2Base model=Seed-Coder-8B, Eff. Tokens (%)=4.62026.06 | 72.4 | 66 | |
| BASEBase model=Qwen2.5-Coder-7B-Instruct2026.06 | 71.6 | — | |
| FULL TOKENSBase model=Qwen2.5-Coder-7B-Instruct, Eff. Tokens (%)=100.02026.06 | 71.4 | — | |
| VCRDTeacher=Qwen2.5-Coder-14B-Inst, Student=Qwen2.5-Coder-7B-Inst2026.04 | 71.2 | 81.75 | |
| CLAMBase model=Seed-Coder-8B, Eff. Tokens (%)=3.62026.06 | 71.1 | 64.9 | |
| DistillLM-2Teacher=Qwen2.5-Coder-14B-Inst, Student=Qwen2.5-Coder-7B-Inst2026.04 | 70.9 | 81.37 | |
| FULL TOKENSBase model=Seed-Coder-8B, Eff. Tokens (%)=100.02026.06 | 70.8 | 65.1 | |
| RANDOM SELECTIONBase model=Seed-Coder-8B, Eff. Tokens (%)=1.92026.06 | 70.6 | 63.9 | |
| DistilLLMTeacher=Qwen2.5-Coder-14B-Inst, Student=Qwen2.5-Coder-7B-Inst2026.04 | 70.4 | 80.75 | |
| CODEBLOCKBase model=Seed-Coder-8B, Eff. Tokens (%)=1.92026.06 | 70.4 | 65.5 | |
| TOKEN CLEANINGBase model=Seed-Coder-8B, Eff. Tokens (%)=1.72026.06 | 69.8 | 62.8 | |
| FULL TOKENSBase model=Qwen2.5-Coder-3B-Instruct, Eff. Tokens (%)=100.02026.06 | 69.6 | — | |
| BASEBase model=Seed-Coder-8B2026.06 | 69 | 62.5 | |
| FULL TOKENSBase model=OpenCoder-8B-Base, Eff. Tokens (%)=100.02026.06 | 68.7 | 54.5 | |
| BASEBase model=OpenCoder-8B-Base2026.06 | 68.5 | 54.4 | |
| RANDOM SELECTIONBase model=OpenCoder-8B-Base, Eff. Tokens (%)=1.92026.06 | 68 | 54.6 | |
| DS2Base model=OpenCoder-8B-Base, Eff. Tokens (%)=4.62026.06 | 68 | 54.6 | |
| DS2Base model=Qwen2.5-Coder-3B-Instruct, Eff. Tokens (%)=4.62026.06 | 68 | — | |
| CODEBLOCKBase model=Qwen2.5-Coder-3B-Instruct, Eff. Tokens (%)=1.92026.06 | 68 | — | |
| CLAMBase model=OpenCoder-8B-Base, Eff. Tokens (%)=3.52026.06 | 67.7 | 54.6 | |
| CLAMBase model=Qwen2.5-Coder-3B-Instruct, Eff. Tokens (%)=3.52026.06 | 67.7 | — | |
| TOKEN CLEANINGBase model=Qwen2.5-Coder-3B-Instruct, Eff. Tokens (%)=1.72026.06 | 67.5 | — | |
| TOKEN CLEANINGBase model=OpenCoder-8B-Base, Eff. Tokens (%)=1.72026.06 | 66.9 | 51.3 | |
| RANDOM SELECTIONBase model=Qwen2.5-Coder-3B-Instruct, Eff. Tokens (%)=1.92026.06 | 66.7 | — | |
| SFTBackbone=Qwen2.5-Coder-3B2026.06 | 66.67 | — | |
| ASFTBackbone=Qwen2.5-Coder-3B2026.06 | 66.4 | — | |
| CODEBLOCKBase model=OpenCoder-8B-Base, Eff. Tokens (%)=1.92026.06 | 66.4 | 57.1 | |
| CLAMBase model=Qwen2.5-Coder-1.5B-Instruct, Eff. Tokens (%)=3.52026.06 | 66.1 | 51.8 | |
| IDFTBackbone=Qwen2.5-Coder-3B2026.06 | 65.87 | — | |
| DFTBackbone=Qwen2.5-Coder-3B2026.06 | 65.34 | — | |
| TALRBackbone=Qwen2.5-Coder-3B2026.06 | 65.34 | — | |
| EAFTBackbone=Qwen2.5-Coder-3B2026.06 | 64.81 | — | |
| RANDOM SELECTIONBase model=Qwen2.5-Coder-1.5B-Instruct, Eff. Tokens (%)=1.92026.06 | 64.6 | 53.6 | |
| DS2Base model=Qwen2.5-Coder-1.5B-Instruct, Eff. Tokens (%)=4.62026.06 | 64.6 | 53.6 | |
| PriFT-massBackbone=Qwen2.5-Coder-3B2026.06 | 64.29 | — | |
| CODEBLOCKBase model=Qwen2.5-Coder-1.5B-Instruct, Eff. Tokens (%)=1.92026.06 | 63.5 | 54.6 | |
| OriginalBackbone=Qwen2.5-Coder-3B2026.06 | 63.49 | — | |
| BASEBase model=Qwen2.5-Coder-3B-Instruct2026.06 | 62.7 | — | |
| PriFT-probBackbone=Qwen2.5-Coder-3B2026.06 | 62.43 | — | |
| FULL TOKENSBase model=Qwen2.5-Coder-1.5B-Instruct, Eff. Tokens (%)=100.02026.06 | 62.4 | 52.7 | |
| BASEBase model=Qwen2.5-Coder-1.5B-Instruct2026.06 | 59.8 | 49.4 | |
| TOKEN CLEANINGBase model=Qwen2.5-Coder-1.5B-Instruct, Eff. Tokens (%)=1.82026.06 | 53.7 | 49.1 | |
| ARES-RLBackbone=Qwen3-32B2026.05 | — | 63.16 | |
| ARES-SFTBackbone=Qwen3-32B2026.05 | — | 61.4 | |
| AWQModel Scale=Llama-3.1-3B, Quantization Bit-width=4-bit, Merging Algorithm=Task Arithmetic2026.05 | — | 56.08 | |
| AWQModel Scale=Llama-3.1-8B, Quantization Bit-width=4-bit, Merging Algorithm=Task Arithmetic2026.05 | — | 60.85 | |
| CPTBackbone=Qwen3-32B2026.05 | — | 59.4 | |
| E-PMQModel Scale=Llama-3.1-3B, Quantization Bit-width=4-bit, Merging Algorithm=Task Arithmetic2026.05 | — | 57.14 | |
| E-PMQModel Scale=Llama-3.1-8B, Quantization Bit-width=4-bit, Merging Algorithm=Task Arithmetic2026.05 | — | 61.38 | |
| GPTQModel Scale=Llama-3.1-3B, Quantization Bit-width=4-bit, Merging Algorithm=Task Arithmetic2026.05 | — | 55.56 | |
| GPTQModel Scale=Llama-3.1-8B, Quantization Bit-width=4-bit, Merging Algorithm=Task Arithmetic2026.05 | — | 57.94 | |
| NaturalReasoningBackbone=Qwen3-32B2026.05 | — | 61.15 | |
| Task ArithmeticModel Scale=Llama-3.1-3B, Quantization Bit-width=Full-precision, Merging Algorithm=Task Arithmetic2026.05 | — | 57.94 | |
| Task ArithmeticModel Scale=Llama-3.1-8B, Quantization Bit-width=Full-precision, Merging Algorithm=Task Arithmetic2026.05 | — | 61.38 | |
| WebscaleBackbone=Qwen3-32B2026.05 | — | 61.4 |