Code Generation on HumanEval (NFE Reduction, TPS Speedup, Accuracy)
6.96NFE ReductionSpiffy
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SpiffyBase Model=SDAR-8B-Chat-b322025.09 | 6.96 | 4.77 | 71 | |
| SpiffyBase Model=LLaDA-8B-Instruct2025.09 | 5.8 | 3.22 | 38 | |
| Dynamic UnmaskingBase Model=SDAR-8B-Chat-b322025.09 | 5.54 | 4.24 | 71 | |
| SpiffyBase Model=Dream-7B-Instruct2025.09 | 4.42 | 2.9 | 55 | |
| Dynamic UnmaskingBase Model=LLaDA-8B-Instruct2025.09 | 3.51 | 2.89 | 41 | |
| Dynamic UnmaskingBase Model=Dream-7B-Instruct2025.09 | 2.68 | 2.73 | 52 | |
| BaselineBase Model=LLaDA-8B-Instruct2025.09 | 1 | 1 | 41 | |
| BaselineBase Model=Dream-7B-Instruct2025.09 | 1 | 1 | 54 | |
| BaselineBase Model=SDAR-8B-Chat-b322025.09 | 1 | 1 | 68 |