Code Generation on HumanEval (TPF, TPS, Speedup, Accuracy)
4.09TPFJacobi Forcing Model (MR)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Jacobi Forcing Model (MR)Family=AR-based, Backbone=Qwen2.5-Coder-7B-Instruct, Multi-block and rejection-recycling=true2025.12 | 4.09 | 163.9 | 3.97 | 83.5 | |
| Jacobi Forcing ModelFamily=AR-based, Backbone=Qwen2.5-Coder-7B-Instruct2025.12 | 4.01 | 159.5 | 3.86 | 83.5 | |
| CLLM*Family=AR-based, Backbone=Qwen2.5-Coder-7B-Instruct, sequence packing=new sequence packing technique2025.12 | 2.68 | 103.3 | 2.5 | 87.8 | |
| D2FFamily=Diffusion-based, Backbone=Dream-7B2025.12 | 2.5 | 73.2 | 1.77 | 54.3 | |
| Fast-dLLM (DC)Family=Diffusion-based, Backbone=Dream-7B, bi-directional dual cache=true2025.12 | 1.8 | 60 | 1.45 | 53 | |
| JacobiFamily=AR-based, Backbone=Qwen2.5-Coder-7B-Instruct2025.12 | 1.03 | 39.9 | 0.97 | 87.8 | |
| ARFamily=AR-based, Backbone=Qwen2.5-Coder-7B-Instruct2025.12 | 1 | 41.3 | 1 | 87.8 | |
| LLaDA-InstructFamily=Diffusion-based, Backbone=LLaDA-7B2025.12 | 1 | 2.8 | 0.07 | 36 | |
| Dream-BaseFamily=Diffusion-based, Backbone=Dream-7B2025.12 | 1 | 20.2 | 0.49 | 54.3 |