Code Generation on HumanEval+ (4-shot)
73.78Functional Correctness ScoreWeDLM-8B
Evaluation Results
| Method | Links | |
|---|---|---|
| WeDLM-8BInstruct Model=true, Model Category=WeDLM, Shot Configuration=4-shot, Inference Framework=vLLM, Sampling Temperature=0.1, Max Generation Length=512, Window Size (W)=6, Penalty Coefficient (λ)=0.12025.12 | 73.78 | |
| WeDLM-7BInstruct Model=true, Model Category=WeDLM, Shot Configuration=4-shot, Inference Framework=vLLM, Sampling Temperature=0.1, Max Generation Length=512, Window Size (W)=6, Penalty Coefficient (λ)=0.12025.12 | 71.34 | |
| Qwen2.5-7BInstruct Model=true, Model Category=AR Baseline, Shot Configuration=4-shot, Inference Framework=vLLM, Sampling Temperature=0.1, Max Generation Length=5122025.12 | 70.12 | |
| SDAR-8BInstruct Model=true, Model Category=DLLM Baseline, Shot Configuration=4-shot, Inference Framework=JetEngine, Sampling Temperature=0.1, Max Generation Length=512, Window Size (W)=6, Penalty Coefficient (λ)=0.12025.12 | 70.12 | |
| Qwen3-8BInstruct Model=true, Model Category=AR Baseline, Shot Configuration=4-shot, Inference Framework=vLLM, Sampling Temperature=0.1, Max Generation Length=5122025.12 | 64.63 | |
| Dream-7BInstruct Model=true, Model Category=DLLM Baseline, Shot Configuration=4-shot, Inference Framework=dInfer, Sampling Temperature=0.1, Max Generation Length=512, Window Size (W)=6, Penalty Coefficient (λ)=0.12025.12 | 51.22 | |
| LLaDA-8BInstruct Model=true, Model Category=DLLM Baseline, Shot Configuration=4-shot, Inference Framework=dInfer, Sampling Temperature=0.1, Max Generation Length=512, Window Size (W)=6, Penalty Coefficient (λ)=0.12025.12 | 32.32 |