Code Generation on HumanEval+ (pass@1, average)
75.6pass@1FULL TOKENS
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FULL TOKENSBase model=Seed-Coder-8B, Eff. Tokens (%)=100.02026.06 | 75.6 | 65.1 | |
| CODEBLOCKBase model=Seed-Coder-8B, Eff. Tokens (%)=1.92026.06 | 75.6 | 65.5 | |
| DS2Base model=Seed-Coder-8B, Eff. Tokens (%)=4.62026.06 | 75 | 66 | |
| CODEBLOCKBase model=Qwen2.5-Coder-1.5B-Instruct, Eff. Tokens (%)=1.92026.06 | 73.8 | 54.6 | |
| RANDOM SELECTIONBase model=Seed-Coder-8B, Eff. Tokens (%)=1.92026.06 | 73.8 | 63.9 | |
| CLAMBase model=Seed-Coder-8B, Eff. Tokens (%)=3.62026.06 | 73.8 | 64.9 | |
| TOKEN CLEANINGBase model=Qwen2.5-Coder-1.5B-Instruct, Eff. Tokens (%)=1.82026.06 | 71.3 | 49.1 | |
| TOKEN CLEANINGBase model=Seed-Coder-8B, Eff. Tokens (%)=1.72026.06 | 71.3 | 62.8 | |
| FULL TOKENSBase model=Qwen2.5-Coder-1.5B-Instruct, Eff. Tokens (%)=100.02026.06 | 69.5 | 52.7 | |
| RANDOM SELECTIONBase model=Qwen2.5-Coder-1.5B-Instruct, Eff. Tokens (%)=1.92026.06 | 69.5 | 53.6 | |
| DS2Base model=Qwen2.5-Coder-1.5B-Instruct, Eff. Tokens (%)=4.62026.06 | 68.9 | 53.6 | |
| CODEBLOCKBase model=OpenCoder-8B-Base, Eff. Tokens (%)=1.92026.06 | 68.9 | 57.1 | |
| BASEBase model=Seed-Coder-8B2026.06 | 68.3 | 62.5 | |
| BASEBase model=Qwen2.5-Coder-1.5B-Instruct2026.06 | 67.1 | 49.4 | |
| CLAMBase model=Qwen2.5-Coder-1.5B-Instruct, Eff. Tokens (%)=3.52026.06 | 62.2 | 51.8 | |
| BASEBase model=OpenCoder-8B-Base2026.06 | 61 | 54.4 | |
| FULL TOKENSBase model=OpenCoder-8B-Base, Eff. Tokens (%)=100.02026.06 | 60.9 | 54.5 | |
| RANDOM SELECTIONBase model=OpenCoder-8B-Base, Eff. Tokens (%)=1.92026.06 | 60.9 | 54.6 | |
| DS2Base model=OpenCoder-8B-Base, Eff. Tokens (%)=4.62026.06 | 60.4 | 54.6 | |
| CLAMBase model=OpenCoder-8B-Base, Eff. Tokens (%)=3.52026.06 | 60.4 | 54.6 | |
| TOKEN CLEANINGBase model=OpenCoder-8B-Base, Eff. Tokens (%)=1.72026.06 | 53.6 | 51.3 |