Code Generation on HumanEval (Accuracy, TPS, Speedup)
81.7AccuracyAdaSD
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| AdaSDDraft-Target Model Pair=Qwen 2.5 72B – Qwen 2.5 7B2025.12 | 81.7 | 16.752 | 1.229 | 0.344 | |
| VanillaDraft-Target Model Pair=Qwen 2.5 72B – Qwen 2.5 7B2025.12 | 79.9 | 13.634 | 1 | — | |
| AssistedGenDraft-Target Model Pair=Qwen 2.5 72B – Qwen 2.5 7B2025.12 | 79.9 | 17.482 | 1.282 | — | |
| Verify-OnlyDraft-Target Model Pair=Llama 3.1 70B – Llama 3.1 8B2025.12 | 79.3 | 14.499 | 1.013 | 0.309 | |
| TargetDraft-Target Model Pair=Qwen 2.5 72B – Qwen 2.5 7B2025.12 | 78.7 | 4.717 | 0.346 | — | |
| Gen-OnlyDraft-Target Model Pair=Qwen 2.5 72B – Qwen 2.5 7B2025.12 | 78.7 | 16.733 | 1.227 | — | |
| Verify-OnlyDraft-Target Model Pair=Qwen 2.5 72B – Qwen 2.5 7B2025.12 | 78 | 13.821 | 1.014 | 0.354 | |
| AssistedGenDraft-Target Model Pair=Llama 3.1 70B – Llama 3.2 1B2025.12 | 77.4 | 23.265 | 1.332 | — | |
| VanillaDraft-Target Model Pair=Llama 3.1 70B – Llama 3.1 8B2025.12 | 76.8 | 14.319 | 1 | — | |
| Gen-OnlyDraft-Target Model Pair=Llama 3.1 70B – Llama 3.2 1B2025.12 | 76.8 | 24.842 | 1.422 | — | |
| Gen-OnlyDraft-Target Model Pair=Llama 3.1 70B – Llama 3.1 8B2025.12 | 76.2 | 18.017 | 1.258 | — | |
| AssistedGenDraft-Target Model Pair=Llama 3.1 70B – Llama 3.1 8B2025.12 | 75.6 | 18.137 | 1.267 | — | |
| VanillaDraft-Target Model Pair=Llama 3.1 70B – Llama 3.2 1B2025.12 | 75.6 | 17.467 | 1 | — | |
| TargetDraft-Target Model Pair=Llama 3.1 70B – Llama 3.1 8B2025.12 | 75 | 4.838 | 0.338 | — | |
| TargetDraft-Target Model Pair=Llama 3.1 70B – Llama 3.2 1B2025.12 | 75 | 4.838 | 0.277 | — | |
| AdaSDDraft-Target Model Pair=Llama 3.1 70B – Llama 3.1 8B2025.12 | 74.4 | 18.839 | 1.316 | 0.313 | |
| AdaSDDraft-Target Model Pair=Llama 3.1 70B – Llama 3.2 1B2025.12 | 73.8 | 26.045 | 1.491 | 0.356 | |
| Verify-OnlyDraft-Target Model Pair=Llama 3.1 70B – Llama 3.2 1B2025.12 | 72.6 | 17.793 | 1.019 | 0.359 | |
| DraftDraft-Target Model Pair=Qwen 2.5 72B – Qwen 2.5 7B2025.12 | 72 | 38.724 | 2.84 | — | |
| DraftDraft-Target Model Pair=Llama 3.1 70B – Llama 3.1 8B2025.12 | 63.4 | 36.257 | 2.532 | — | |
| DKV-CacheModel Variant=Base2026.01 | 59.8 | 11.1 | 1.3 | — | |
| Window-DiffusionModel Variant=Instruct, Window Size (L)=16, Refresh Cycle=32, Early Stopping=Disabled2026.01 | 58.5 | 32.3 | 5.3 | — | |
| DreamModel Variant=Base2026.01 | 57.9 | 8.6 | 1 | — | |
| Fast-dLLMModel Variant=Base, Caching Strategy=Dual-Cache2026.01 | 57.9 | 27.5 | 3.2 | — | |
| Window-DiffusionModel Variant=Base, Window Size (L)=16, Refresh Cycle=32, Early Stopping=Disabled2026.01 | 57.9 | 33.7 | 3.9 | — | |
| ConfidenceModel=Dream-v0-Instruct-7B2026.02 | 57.32 | 52.11 | 2.3 | — | |
| DreamModel Variant=Instruct2026.01 | 55.5 | 6.1 | 1 | — | |
| Fast-dLLMModel Variant=Base, Caching Strategy=Prefix-Cache2026.01 | 54.9 | 17.4 | 2 | — | |
| Fast-dLLMModel Variant=Instruct, Caching Strategy=Prefix-Cache2026.01 | 54.9 | 12.9 | 2.1 | — | |
| KLASSModel=Dream-v0-Instruct-7B2026.02 | 54.88 | 26.71 | 1.18 | — | |
| DAWNModel=Dream-v0-Instruct-7B2026.02 | 54.88 | 60.23 | 2.66 | — | |
| LocalLeapModel=Dream-v0-Instruct-7B2026.02 | 54.27 | 59.28 | 2.62 | — | |
| OriginalModel=Dream-v0-Instruct-7B2026.02 | 53.66 | 22.66 | 1 | — | |
| OriginalModel=LLaDA-1.52026.02 | 43.9 | 9.26 | 1 | — | |
| ConfidenceModel=LLaDA-1.52026.02 | 42.68 | 23.39 | 2.53 | — | |
| KLASSModel=Dream-v0-Base-7B2026.02 | 42.68 | 31.32 | 1.5 | — | |
| LocalLeapModel=LLaDA-1.52026.02 | 42.07 | 30.21 | 3.26 | — | |
| DAWNModel=LLaDA-1.52026.02 | 42.07 | 29.53 | 3.19 | — | |
| ConfidenceModel=LLaDA-8B-Instruct2026.02 | 40.85 | 86.73 | 3.28 | — | |
| OriginalModel=LLaDA-8B-Instruct2026.02 | 40.24 | 26.46 | 1 | — | |
| DAWNModel=LLaDA-8B-Instruct2026.02 | 40.24 | 108.99 | 4.12 | — | |
| LocalLeapModel=LLaDA-8B-Instruct2026.02 | 39.63 | 109.8 | 4.15 | — | |
| KLASSModel=LLaDA-1.52026.02 | 39.63 | 16.49 | 1.78 | — | |
| ConfidenceModel=Dream-v0-Base-7B2026.02 | 39.63 | 61.54 | 2.94 | — | |
| DAWNModel=Dream-v0-Base-7B2026.02 | 39.63 | 68.96 | 3.29 | — | |
| OriginalModel=Dream-v0-Base-7B2026.02 | 38.41 | 20.9 | 1 | — | |
| KLASSModel=LLaDA-8B-Instruct2026.02 | 36.59 | 56.91 | 2.15 | — | |
| LocalLeapModel=Dream-v0-Base-7B2026.02 | 36.59 | 68.28 | 3.27 | — | |
| DraftDraft-Target Model Pair=Llama 3.1 70B – Llama 3.2 1B2025.12 | 34.8 | 100.683 | 5.764 | — | |
| DKV-CacheModel Variant=Instruct2026.01 | 33.5 | 8.1 | 1.3 | — | |
| Fast-dLLMModel Variant=Instruct, Caching Strategy=Dual-Cache2026.01 | 23.8 | 25.9 | 4.2 | — |