Program Synthesis on HumanEval
72.15AccuracyHILA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| HILAType=MA, Backbone=LLaMA3-8B2026.03 | 72.15 | — | — | |
| AFlowType=MA, Backbone=LLaMA3-8B2026.03 | 62.2 | — | — | |
| DyLANType=MA, Backbone=LLaMA3-8B2026.03 | 61.59 | — | — | |
| G-SwarmType=MA, Backbone=LLaMA3-8B2026.03 | 59.55 | — | — | |
| KL divergenceParallel=Yes, Backbone=Dream 7B Instruct2025.11 | 59.35 | — | 73.94 | |
| KLASSParallel=Yes, Backbone=Dream 7B Instruct2025.11 | 59.35 | — | 74.88 | |
| Top-1Parallel=No, Backbone=Dream 7B Instruct2025.11 | 58.53 | — | 256 | |
| G-DebateType=MA, Backbone=LLaMA3-8B2026.03 | 57.93 | — | — | |
| DebateType=MA, Backbone=LLaMA3-8B2026.03 | 57.72 | — | — | |
| SCType=SA, Backbone=LLaMA3-8B2026.03 | 57.52 | — | — | |
| A-PruneType=MA, Backbone=LLaMA3-8B2026.03 | 57.11 | — | — | |
| PHPType=MA, Backbone=LLaMA3-8B2026.03 | 56.5 | — | — | |
| WavefrontDiffusionModel Family=Dream-7B2025.11 | 54.27 | 101.4 | — | |
| BlockDiffusionModel Family=Dream-7B2025.11 | 53.05 | 100.6 | — | |
| CoTType=SA, Backbone=LLaMA3-8B2026.03 | 51.42 | — | — | |
| ConfidenceParallel=Yes, Backbone=Dream 7B Instruct2025.11 | 50 | — | 52.47 | |
| WavefrontDiffusionModel Family=LLaDA-1.52025.11 | 48.17 | 112.9 | — | |
| WavefrontDiffusionModel Family=LLaDA-8B-Instruct2025.11 | 47.56 | 109.3 | — | |
| VanillaType=SA, Backbone=LLaMA3-8B2026.03 | 47.56 | — | — | |
| BlockDiffusionModel Family=LLaDA-1.52025.11 | 46.34 | 110.1 | — | |
| BlockDiffusionModel Family=LLaDA-8B-Instruct2025.11 | 45.73 | 107.2 | — | |
| Top-2Parallel=Yes, Backbone=Dream 7B Instruct2025.11 | 42.88 | — | 128 | |
| KLASSParallel=Yes, Backbone=LLaDA 8B Instruct2025.11 | 40.85 | — | 91.98 | |
| KL divergenceParallel=Yes, Backbone=LLaDA 8B Instruct2025.11 | 40.24 | — | 111.93 | |
| Top-1Parallel=No, Backbone=LLaDA 8B Instruct2025.11 | 39.63 | — | 256 | |
| ConfidenceParallel=Yes, Backbone=LLaDA 8B Instruct2025.11 | 37.8 | — | 54.41 | |
| Standard DiffusionModel Family=LLaDA-1.52025.11 | 34.76 | 118 | — | |
| Top-2Parallel=Yes, Backbone=LLaDA 8B Instruct2025.11 | 33.54 | — | 128 | |
| RandomParallel=No, Backbone=LLaDA 8B Instruct2025.11 | 20.21 | — | 256 | |
| Standard DiffusionModel Family=Dream-7B2025.11 | 20.12 | 99.5 | — | |
| RandomParallel=No, Backbone=Dream 7B Instruct2025.11 | 18.09 | — | 256 | |
| Standard DiffusionModel Family=LLaDA-8B-Instruct2025.11 | 17.68 | 115.8 | — |