Code Generation on HumanEval (Performance and Accuracy Profile)
222.68TPST3D
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| T3DDecoding Strategy=Dynamic, Block Size=4, Steps Per Block=4, Confidence Threshold=0.9, Temperature=0.1, Model=SDAR-4B-Chat2026.02 | 222.68 | 0.26 | 16.21 | 58.1 | 29.27 | |
| Naive TDDecoding Strategy=Dynamic, Block Size=4, Steps Per Block=4, Confidence Threshold=0.9, Temperature=0.1, Model=SDAR-4B-Chat2026.02 | 216.39 | 0.29 | 17.15 | 62.1 | 23.17 | |
| OriginalDecoding Strategy=Dynamic, Block Size=4, Steps Per Block=4, Confidence Threshold=0.9, Temperature=0.1, Model=SDAR-4B-Chat2026.02 | 175.48 | 0.73 | 36.56 | 127.54 | 33.54 | |
| ReDiDecoding Strategy=Dynamic, Block Size=4, Steps Per Block=4, Confidence Threshold=0.9, Temperature=0.1, Model=SDAR-4B-Chat2026.02 | 163.77 | 0.47 | 21.23 | 76.75 | 10 | |
| dParallelDecoding Strategy=Dynamic, Block Size=4, Steps Per Block=4, Confidence Threshold=0.9, Temperature=0.1, Model=SDAR-4B-Chat2026.02 | 130.34 | 0.48 | 17.41 | 62.19 | 23.78 | |
| DSB (const.)Backbone=LLaDA-8B-Instruct, Decode=Confidence, Cache=None, Block=DSB (const.)2026.02 | 124.6 | — | — | — | 42.07 | |
| DSB (greedy)Backbone=LLaDA-8B-Instruct, Decode=Confidence, Cache=None, Block=DSB (greedy)2026.02 | 123.8 | — | — | — | 39.63 | |
| Confidence + NaiveBackbone=LLaDA-8B-Instruct, Decode=Confidence, Cache=None, Block=Naive2026.02 | 119.1 | — | — | — | 40.85 | |
| DSB Cache (greedy)Backbone=LLaDA-8B-Instruct, Decode=Confidence, Cache=DSB, Block=DSB (greedy)2026.02 | 107.7 | — | — | — | 39.63 | |
| DSB Cache (const.)Backbone=LLaDA-8B-Instruct, Decode=Confidence, Cache=DSB, Block=DSB (const.)2026.02 | 105.3 | — | — | — | 37.8 | |
| Dual NaiveBackbone=LLaDA-8B-Instruct, Decode=Confidence, Cache=Dual, Block=Naive2026.02 | 100.6 | — | — | — | 37.8 | |
| Confidence + NaiveBackbone=Dream-v0-Base-7B, Decode=Confidence, Cache=None, Block=Naive2026.02 | 91.64 | — | — | — | 39.02 | |
| DSB (const.)Backbone=Dream-v0-Base-7B, Decode=Confidence, Cache=None, Block=DSB (const.)2026.02 | 91.41 | — | — | — | 39.02 | |
| DSB (greedy)Backbone=Dream-v0-Base-7B, Decode=Confidence, Cache=None, Block=DSB (greedy)2026.02 | 88.62 | — | — | — | 39.02 | |
| DSB Cache (const.)Backbone=Dream-v0-Base-7B, Decode=Confidence, Cache=DSB, Block=DSB (const.)2026.02 | 78.09 | — | — | — | 42.07 | |
| Confidence + NaiveBackbone=Dream-v0-Instruct-7B, Decode=Confidence, Cache=None, Block=Naive2026.02 | 77.61 | — | — | — | 57.32 | |
| DSB (const.)Backbone=Dream-v0-Instruct-7B, Decode=Confidence, Cache=None, Block=DSB (const.)2026.02 | 77.04 | — | — | — | 57.32 | |
| DSB (greedy)Backbone=Dream-v0-Instruct-7B, Decode=Confidence, Cache=None, Block=DSB (greedy)2026.02 | 74.58 | — | — | — | 58.54 | |
| Dual NaiveBackbone=Dream-v0-Base-7B, Decode=Confidence, Cache=Dual, Block=Naive2026.02 | 74.49 | — | — | — | 36.59 | |
| DSB Cache (greedy)Backbone=Dream-v0-Base-7B, Decode=Confidence, Cache=DSB, Block=DSB (greedy)2026.02 | 73.11 | — | — | — | 38.41 | |
| DSB Cache (const.)Backbone=Dream-v0-Instruct-7B, Decode=Confidence, Cache=DSB, Block=DSB (const.)2026.02 | 66.01 | — | — | — | 52.44 | |
| Dual NaiveBackbone=Dream-v0-Instruct-7B, Decode=Confidence, Cache=Dual, Block=Naive2026.02 | 65.91 | — | — | — | 48.78 | |
| DSB Cache (greedy)Backbone=Dream-v0-Instruct-7B, Decode=Confidence, Cache=DSB, Block=DSB (greedy)2026.02 | 65.28 | — | — | — | 57.32 | |
| VanillaBackbone=Dream-v0-Base-7B, Decode=Vanilla, Cache=None, Block=Naive2026.02 | 49.77 | — | — | — | 38.41 | |
| VanillaBackbone=LLaDA-8B-Instruct, Decode=Vanilla, Cache=None, Block=Naive2026.02 | 36.22 | — | — | — | 40.24 | |
| DSB (greedy)Backbone=LLaDA-1.5, Decode=Confidence, Cache=None, Block=DSB (greedy)2026.02 | 33.75 | — | — | — | 42.7 | |
| VanillaBackbone=Dream-v0-Instruct-7B, Decode=Vanilla, Cache=None, Block=Naive2026.02 | 33.47 | — | — | — | 53.05 | |
| DSB (const.)Backbone=LLaDA-1.5, Decode=Confidence, Cache=None, Block=DSB (const.)2026.02 | 32.71 | — | — | — | 43.3 | |
| Confidence + NaiveBackbone=LLaDA-1.5, Decode=Confidence, Cache=None, Block=Naive2026.02 | 32.67 | — | — | — | 43.3 | |
| DSB Cache (const.)Backbone=LLaDA-1.5, Decode=Confidence, Cache=DSB, Block=DSB (const.)2026.02 | 32.4 | — | — | — | 35.98 | |
| DSB Cache (greedy)Backbone=LLaDA-1.5, Decode=Confidence, Cache=DSB, Block=DSB (greedy)2026.02 | 31.92 | — | — | — | 34.7 | |
| Dual NaiveBackbone=LLaDA-1.5, Decode=Confidence, Cache=Dual, Block=Naive2026.02 | 30.91 | — | — | — | 34.1 | |
| d2CacheBackbone=Dream-Inst, n-shot=0-shot, Generation Length=5122025.09 | 14.06 | 36.61 | — | — | 61.6 | |
| d2CacheBackbone=LLaDA-Inst, n-shot=0-shot, Generation Length=5122025.09 | 14 | 35.44 | — | — | 48.2 | |
| VanillaBackbone=LLaDA-1.5, Decode=Vanilla, Cache=None, Block=Naive2026.02 | 12.78 | — | — | — | 44.5 | |
| dLLM-CacheBackbone=LLaDA-Inst, n-shot=0-shot, Generation Length=5122025.09 | 8.67 | 57.48 | — | — | 44.5 | |
| Fast-dLLMBackbone=LLaDA-Inst, n-shot=0-shot, Generation Length=5122025.09 | 7.9 | 63.12 | — | — | 43.9 | |
| Fast-dLLMBackbone=Dream-Inst, n-shot=0-shot, Generation Length=5122025.09 | 7.89 | 63.84 | — | — | 56.1 | |
| dLLM-CacheBackbone=Dream-Inst, n-shot=0-shot, Generation Length=5122025.09 | 5.35 | 94.33 | — | — | 56.5 | |
| VanillaBackbone=LLaDA-Inst, n-shot=0-shot, Generation Length=5122025.09 | 4.99 | 105.76 | — | — | 45.1 | |
| VanillaBackbone=Dream-Inst, n-shot=0-shot, Generation Length=5122025.09 | 4.39 | 114.86 | — | — | 56.7 |