Code Generation on HumanEval (pass@1, average)
81.1pass@1CODEBLOCK
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CODEBLOCKBase model=Seed-Coder-8B, Eff. Tokens (%)=1.92026.06 | 81.1 | 65.5 | |
| FULL TOKENSBase model=Seed-Coder-8B, Eff. Tokens (%)=100.02026.06 | 80.5 | 65.1 | |
| DS2Base model=Seed-Coder-8B, Eff. Tokens (%)=4.62026.06 | 80.5 | 66 | |
| CLAMBase model=Seed-Coder-8B, Eff. Tokens (%)=3.62026.06 | 79.2 | 64.9 | |
| TOKEN CLEANINGBase model=Qwen2.5-Coder-1.5B-Instruct, Eff. Tokens (%)=1.82026.06 | 78.7 | 49.1 | |
| CODEBLOCKBase model=Qwen2.5-Coder-1.5B-Instruct, Eff. Tokens (%)=1.92026.06 | 78.7 | 54.6 | |
| BASEBase model=Seed-Coder-8B2026.06 | 78.7 | 62.5 | |
| RANDOM SELECTIONBase model=Seed-Coder-8B, Eff. Tokens (%)=1.92026.06 | 78.7 | 63.9 | |
| RANDOM SELECTIONBase model=Qwen2.5-Coder-1.5B-Instruct, Eff. Tokens (%)=1.92026.06 | 76.2 | 53.6 | |
| CODEBLOCKBase model=OpenCoder-8B-Base, Eff. Tokens (%)=1.92026.06 | 76.2 | 57.1 | |
| FULL TOKENSBase model=Qwen2.5-Coder-1.5B-Instruct, Eff. Tokens (%)=100.02026.06 | 75.6 | 52.7 | |
| DS2Base model=Qwen2.5-Coder-1.5B-Instruct, Eff. Tokens (%)=4.62026.06 | 75 | 53.6 | |
| TOKEN CLEANINGBase model=Seed-Coder-8B, Eff. Tokens (%)=1.72026.06 | 75 | 62.8 | |
| BASEBase model=Qwen2.5-Coder-1.5B-Instruct2026.06 | 71.3 | 49.4 | |
| CLAMBase model=Qwen2.5-Coder-1.5B-Instruct, Eff. Tokens (%)=3.52026.06 | 67.1 | 51.8 | |
| BASEBase model=OpenCoder-8B-Base2026.06 | 67.1 | 54.4 | |
| RANDOM SELECTIONBase model=OpenCoder-8B-Base, Eff. Tokens (%)=1.92026.06 | 65.9 | 54.6 | |
| DS2Base model=OpenCoder-8B-Base, Eff. Tokens (%)=4.62026.06 | 65.9 | 54.6 | |
| CLAMBase model=OpenCoder-8B-Base, Eff. Tokens (%)=3.52026.06 | 65.2 | 54.6 | |
| FULL TOKENSBase model=OpenCoder-8B-Base, Eff. Tokens (%)=100.02026.06 | 64 | 54.5 | |
| TOKEN CLEANINGBase model=OpenCoder-8B-Base, Eff. Tokens (%)=1.72026.06 | 59.1 | 51.3 |