Code Generation on MBPP 3-shot (Accuracy, TPS, Steps, Latency)
59.6AccuracyKLASS
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| KLASSBase Model=Dream-7B-Instruct2026.05 | 59.6 | 9.2 | 54 | 9.1 | — | |
| KLASSBackbone Model=Dream-v0-Base-7B, Shot Setting=3-shot2026.05 | 56.43 | 8.12 | — | 0.88 | — | |
| KLASSModel=Dream-v0-Instruct-7B2026.05 | 56.2 | 9.44 | — | 2.52 | — | |
| Fast-dLLMModel=Dream-v0-Instruct-7B2026.05 | 56 | 17.71 | — | 4.74 | — | |
| DAWNModel=Dream-v0-Instruct-7B2026.05 | 56 | 20.15 | — | 5.39 | — | |
| ThresholdModel=Dream-v0-Base-7B, PrefixCache=Enabled, Block size=32, Hardware=H100, Len.=2562026.06 | 56 | 85.1 | 47,070 | — | 2.72 | |
| BaselineBase Model=Dream-7B-Instruct2026.05 | 55 | 1 | 512 | 1 | — | |
| Conf-BasedBase Model=Dream-7B-Instruct2026.05 | 54.8 | 17.8 | 27 | 18.1 | — | |
| VanillaModel=Dream-v0-Instruct-7B2026.05 | 54.8 | 3.74 | — | 1 | — | |
| LoPABase Model=Dream-7B-Instruct2026.05 | 54.6 | 10 | 34 | 10.1 | — | |
| DAPDModel=Dream-v0-Instruct-7B2026.05 | 54.4 | 5.35 | — | 1.43 | — | |
| CLADBackbone Model=Dream-v0-Base-7B, Shot Setting=3-shot2026.05 | 54.2 | 29.06 | — | 3.17 | — | |
| CLADModel=Dream-v0-Instruct-7B2026.05 | 54 | 20.64 | — | 5.52 | — | |
| DAWNBackbone Model=Dream-v0-Base-7B, Shot Setting=3-shot2026.05 | 53.73 | 33.2 | — | 3.62 | — | |
| Fast-dLLMBackbone Model=Dream-v0-Base-7B, Shot Setting=3-shot2026.05 | 53.2 | 25.35 | — | 2.76 | — | |
| FrechetModel=Dream-v0-Base-7B, PrefixCache=Enabled, Block size=32, Hardware=H100, Len.=2562026.06 | 53.2 | 111.2 | 35,170 | — | 3.64 | |
| ThresholdModel=Dream-v0-Base-7B, PrefixCache=Enabled, Block size=32, Hardware=H100, Len.=5122026.06 | 53.2 | 117.6 | 63,108 | — | 4.06 | |
| LEAPBase Model=Dream-7B-Instruct2026.05 | 53 | 25.2 | 18 | 25 | — | |
| VanillaBackbone Model=Dream-v0-Base-7B, Shot Setting=3-shot2026.05 | 52.8 | 9.18 | — | 1 | — | |
| FactorModel=Dream-v0-Base-7B, PrefixCache=Enabled, Block size=32, Hardware=H100, Len.=5122026.06 | 52.8 | 141.5 | 51,779 | — | 4.94 | |
| FrechetModel=Dream-v0-Base-7B, PrefixCache=Enabled, Block size=32, Hardware=H100, Len.=5122026.06 | 52.8 | 153.2 | 47,709 | — | 5.37 | |
| FactorModel=Dream-v0-Base-7B, PrefixCache=Enabled, Block size=32, Hardware=H100, Len.=2562026.06 | 51.8 | 106.6 | 38,043 | — | 3.43 | |
| DAPDBackbone Model=Dream-v0-Base-7B, Shot Setting=3-shot2026.05 | 49.2 | 17.02 | — | 1.85 | — | |
| d3LLM-TF-LLaDABlock size=2562026.06 | 40.8 | 84.37 | — | — | — | |
| KLASSBase Model=LLaDA-8B-Instruct2026.05 | 40.2 | 15.2 | 67 | 7.6 | — | |
| LLaDABlock size=2562026.06 | 40.2 | 15.67 | — | — | — | |
| AsyncLane-LLaDABlock size=5122026.06 | 39.8 | 106.26 | — | — | — | |
| d3LLM-TF-LLaDABlock size=5122026.06 | 39.6 | 76.2 | — | — | — | |
| LLaDABlock size=10242026.06 | 39.4 | 2.85 | — | — | — | |
| LLaDABlock size=5122026.06 | 39 | 10.76 | — | — | — | |
| CLADBackbone Model=LLaDA-1.5, Shot Setting=3-shot2026.05 | 38.8 | 16.86 | — | 8.47 | — | |
| VanillaBackbone Model=LLaDA-1.5, Shot Setting=3-shot2026.05 | 38.6 | 1.99 | — | 1 | — | |
| DAPDBackbone Model=LLaDA-1.5, Shot Setting=3-shot2026.05 | 38.6 | 3.89 | — | 1.95 | — | |
| Fast-dLLM-LLaDABlock size=5122026.06 | 38.6 | 77.38 | — | — | — | |
| AsyncLane-LLaDABlock size=2562026.06 | 38.4 | 112.43 | — | — | — | |
| Fast-dLLMBackbone Model=LLaDA-1.5, Shot Setting=3-shot2026.05 | 38 | 11.78 | — | 5.92 | — | |
| Fast-dLLM-LLaDABlock size=2562026.06 | 37.8 | 76.69 | — | — | — | |
| AsyncLane-LLaDABlock size=10242026.06 | 37.8 | 111.89 | — | — | — | |
| DAWNBackbone Model=LLaDA-1.5, Shot Setting=3-shot2026.05 | 37.4 | 16.27 | — | 8.18 | — | |
| BaselineBase Model=LLaDA-8B-Instruct2026.05 | 37 | 1 | 512 | 1 | — | |
| LEAPBase Model=LLaDA-8B-Instruct2026.05 | 37 | 21.6 | 22 | 20.3 | — | |
| Conf-BasedBase Model=LLaDA-8B-Instruct2026.05 | 36.2 | 11.9 | 45 | 11.3 | — | |
| d3LLM-TF-LLaDABlock size=10242026.06 | 36 | 52.93 | — | — | — | |
| Fast-dLLM-LLaDABlock size=10242026.06 | 35.8 | 53.09 | — | — | — | |
| DAPDModel=LLaDA-8B-Instruct2026.05 | 32 | 8.08 | — | 1.48 | — | |
| KLASSModel=LLaDA-8B-Instruct2026.05 | 31.96 | 15.62 | — | 2.87 | — | |
| Fast-dLLMModel=LLaDA-8B-Instruct2026.05 | 31.8 | 20.01 | — | 3.67 | — | |
| VanillaModel=LLaDA-8B-Instruct2026.05 | 31.6 | 5.45 | — | 1 | — | |
| DAWNModel=LLaDA-8B-Instruct2026.05 | 30.6 | 26.53 | — | 4.87 | — | |
| CLADModel=LLaDA-8B-Instruct2026.05 | 30.2 | 26.66 | — | 4.89 | — | |
| ThresholdGen. Len.=256, Backbone=LLaDA-8B-Instruct, Threshold=τ = 0.92026.06 | 27.4 | 66 | 34,975 | — | 2.41 | |
| KLASSBackbone Model=LLaDA-1.5, Shot Setting=3-shot2026.05 | 25.8 | 1.45 | — | 0.73 | — | |
| Fréchet profile decodingGen. Len.=256, Backbone=LLaDA-8B-Instruct, Margin=δ = 0.252026.06 | 25.4 | 85.4 | 25,791 | — | 3.34 | |
| FactorGen. Len.=256, Backbone=LLaDA-8B-Instruct2026.06 | 21.4 | 81.6 | 26,870 | — | 3.16 | |
| ThresholdGen. Len.=512, Backbone=LLaDA-8B-Instruct, Threshold=τ = 0.92026.06 | 14.2 | 60.8 | 59,999 | — | 2.5 | |
| Fréchet profile decodingGen. Len.=512, Backbone=LLaDA-8B-Instruct, Margin=δ = 0.252026.06 | 14.2 | 82.7 | 42,893 | — | 3.49 | |
| FactorGen. Len.=512, Backbone=LLaDA-8B-Instruct2026.06 | 12 | 80 | 45,186 | — | 3.33 |