Code Generation on HumanEval (Accuracy, Steps, Speed)
63.8Accuracy (%)Info-Gain
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Info-GainModel=TraDo-8B-Instruct, K (Decoding Rate)=12026.02 | 63.8 | — | — | |
| Info-GainModel=TraDo-8B-Instruct, K (Decoding Rate)=22026.02 | 58.8 | — | — | |
| COVERBackbone=Dream-Ins-7B, Length=5122026.02 | 57.93 | 71.27 | 5.78 | |
| BaseBackbone=Qwen2.5-Math-7B, Temperature (τ)=0.102026.02 | 56.71 | — | — | |
| BoKBackbone=Qwen2.5-Math-7B, Temperature (τ)=0.10, β=0.05, λ=0.52026.02 | 56.01 | — | — | |
| COVERBackbone=Dream-Ins-7B, Length=2562026.02 | 55.49 | 71.56 | 2.77 | |
| baselineBackbone=Dream-Ins-7B, Length=2562026.02 | 54.88 | 256 | 1 | |
| SaberBackbone=Dream-Ins-7B, Length=2562026.02 | 54.88 | 186.16 | 1.45 | |
| WINOBackbone=Dream-Ins-7B, Length=2562026.02 | 54.88 | 83.76 | 2.7 | |
| Top-KBackbone=Qwen2.5-Math-7B, Temperature (τ)=0.10, K=502026.02 | 54.88 | — | — | |
| BoKBackbone=Qwen2.5-Math-7B, Temperature (τ)=0.25, β=0.01, λ=0.12026.02 | 54.88 | — | — | |
| BoKBackbone=Qwen2.5-Math-7B, Temperature (τ)=0.10, β=0.02, λ=0.22026.02 | 54.88 | — | — | |
| BoKBackbone=Qwen2.5-Math-7B, Temperature (τ)=0.25, β=0.02, λ=0.22026.02 | 54.88 | — | — | |
| baselineBackbone=Dream-Ins-7B, Length=5122026.02 | 54.27 | 512 | 1 | |
| BoKBackbone=Qwen2.5-Math-7B, Temperature (τ)=0.10, β=0.01, λ=0.12026.02 | 54.27 | — | — | |
| BoKBackbone=Qwen2.5-Math-7B, Temperature (τ)=0.70, β=0.01, λ=0.12026.02 | 54.27 | — | — | |
| WINOBackbone=Dream-Ins-7B, Length=5122026.02 | 53.05 | 99.37 | 4.73 | |
| BaseBackbone=Qwen2.5-Math-7B, Temperature (τ)=0.252026.02 | 53.05 | — | — | |
| BoKBackbone=Qwen2.5-Math-7B, Temperature (τ)=0.70, β=0.02, λ=0.22026.02 | 53.05 | — | — | |
| SaberBackbone=Dream-Ins-7B, Length=5122026.02 | 52.44 | 448.85 | 1.24 | |
| BoKBackbone=Qwen2.5-Math-7B, Temperature (τ)=0.90, β=0.01, λ=0.12026.02 | 52.44 | — | — | |
| BoKBackbone=Qwen2.5-Math-7B, Temperature (τ)=0.90, β=0.02, λ=0.22026.02 | 52.44 | — | — | |
| BoKBackbone=Qwen2.5-Math-7B, Temperature (τ)=0.70, β=0.05, λ=0.52026.02 | 52.44 | — | — | |
| Top-KBackbone=Qwen2.5-Math-7B, Temperature (τ)=0.25, K=502026.02 | 51.83 | — | — | |
| Top-KBackbone=Qwen2.5-Math-7B, Temperature (τ)=0.50, K=502026.02 | 51.83 | — | — | |
| BoKBackbone=Qwen2.5-Math-7B, Temperature (τ)=0.50, β=0.02, λ=0.22026.02 | 51.83 | — | — | |
| BoKBackbone=Qwen2.5-Math-7B, Temperature (τ)=0.25, β=0.05, λ=0.52026.02 | 51.83 | — | — | |
| BoKBackbone=Qwen2.5-Math-7B, Temperature (τ)=0.50, β=0.05, λ=0.52026.02 | 51.83 | — | — | |
| Info-GainModel=SDAR-8B-Chat, K (Decoding Rate)=12026.02 | 51.3 | — | — | |
| BoKBackbone=Qwen2.5-Math-7B, Temperature (τ)=0.90, β=0.05, λ=0.52026.02 | 51.22 | — | — | |
| BoKBackbone=Qwen2.5-Math-7B, Temperature (τ)=0.50, β=0.01, λ=0.12026.02 | 50 | — | — | |
| BaseBackbone=Qwen2.5-Math-7B, Temperature (τ)=0.702026.02 | 49.39 | — | — | |
| baselineBackbone=LLaDA-1.5-8B, Length=5122026.02 | 48.78 | 512 | 1 | |
| WINOBackbone=LLaDA-1.5-8B, Length=5122026.02 | 48.78 | 205.16 | 1.39 | |
| COVERBackbone=LLaDA-1.5-8B, Length=5122026.02 | 48.78 | 140.44 | 2.66 | |
| BaseBackbone=Qwen2.5-Math-7B, Temperature (τ)=0.502026.02 | 48.78 | — | — | |
| COVERBackbone=LLaDA-Ins-8B, Length=5122026.02 | 48.17 | 132.63 | 2.72 | |
| SaberBackbone=LLaDA-1.5-8B, Length=5122026.02 | 48.17 | 225.11 | 2.22 | |
| SaberBackbone=LLaDA-Ins-8B, Length=5122026.02 | 47.56 | 191.29 | 2.58 | |
| WINOBackbone=LLaDA-Ins-8B, Length=5122026.02 | 47.56 | 178.44 | 1.59 | |
| baselineBackbone=LLaDA-Ins-8B, Length=5122026.02 | 46.95 | 512 | 1 | |
| Top-KBackbone=Qwen2.5-Math-7B, Temperature (τ)=0.70, K=502026.02 | 46.34 | — | — | |
| Info-GainModel=SDAR-8B-Chat, K (Decoding Rate)=22026.02 | 46.3 | — | — | |
| baselineBackbone=LLaDA-1.5-8B, Length=2562026.02 | 43.29 | 256 | 1 | |
| COVERBackbone=LLaDA-1.5-8B, Length=2562026.02 | 43.29 | 75.09 | 2.44 | |
| WINOBackbone=LLaDA-1.5-8B, Length=2562026.02 | 42.07 | 83.93 | 2.41 | |
| COVERBackbone=LLaDA-Ins-8B, Length=2562026.02 | 41.46 | 53.68 | 3.51 | |
| SaberBackbone=LLaDA-1.5-8B, Length=2562026.02 | 40.85 | 100.93 | 2.38 | |
| WINOBackbone=LLaDA-Ins-8B, Length=2562026.02 | 39.63 | 62.44 | 3.28 | |
| Top-KBackbone=Qwen2.5-Math-7B, Temperature (τ)=0.90, K=502026.02 | 37.8 | — | — | |
| baselineBackbone=LLaDA-Ins-8B, Length=2562026.02 | 37.2 | 256 | 1 | |
| SaberBackbone=LLaDA-Ins-8B, Length=2562026.02 | 37.2 | 93.44 | 2.61 | |
| KLASSModel=TraDo-8B-Instruct, K (Decoding Rate)=12026.02 | 35.7 | — | — | |
| COVERBackbone=LLaDA-Base-8B, Length=5122026.02 | 35.37 | 70.9 | 4.64 | |
| COVERBackbone=LLaDA-Base-8B, Length=2562026.02 | 34.76 | 46.4 | 2.98 | |
| WINOBackbone=LLaDA-Base-8B, Length=5122026.02 | 34.76 | 103.18 | 2.75 | |
| KLASSModel=SDAR-8B-Chat, K (Decoding Rate)=12026.02 | 34.7 | — | — | |
| baselineBackbone=LLaDA-Base-8B, Length=2562026.02 | 33.54 | 256 | 1 | |
| WINOBackbone=LLaDA-Base-8B, Length=2562026.02 | 33.54 | 71.43 | 2.19 | |
| SaberBackbone=LLaDA-Base-8B, Length=5122026.02 | 33.54 | 114.81 | 4.45 | |
| SaberBackbone=LLaDA-Base-8B, Length=2562026.02 | 32.93 | 93.75 | 2.67 | |
| baselineBackbone=LLaDA-Base-8B, Length=5122026.02 | 32.93 | 512 | 1 | |
| BaseBackbone=Qwen2.5-Math-7B, Temperature (τ)=0.902026.02 | 32.93 | — | — | |
| KLASSModel=SDAR-8B-Chat, K (Decoding Rate)=22026.02 | 30.7 | — | — | |
| KLASSModel=TraDo-8B-Instruct, K (Decoding Rate)=22026.02 | 30.7 | — | — | |
| EntropyModel=SDAR-8B-Chat, K (Decoding Rate)=12026.02 | 30.2 | — | — | |
| ConfidenceModel=SDAR-8B-Chat, K (Decoding Rate)=12026.02 | 28.4 | — | — | |
| EntropyModel=SDAR-8B-Chat, K (Decoding Rate)=22026.02 | 26.2 | — | — | |
| EntropyModel=TraDo-8B-Instruct, K (Decoding Rate)=12026.02 | 25.7 | — | — | |
| MarginModel=SDAR-8B-Chat, K (Decoding Rate)=12026.02 | 24.5 | — | — | |
| ConfidenceModel=TraDo-8B-Instruct, K (Decoding Rate)=12026.02 | 24.5 | — | — | |
| ConfidenceModel=SDAR-8B-Chat, K (Decoding Rate)=22026.02 | 24.4 | — | — | |
| MarginModel=TraDo-8B-Instruct, K (Decoding Rate)=12026.02 | 23.9 | — | — | |
| EntropyModel=TraDo-8B-Instruct, K (Decoding Rate)=22026.02 | 20.7 | — | — | |
| MarginModel=SDAR-8B-Chat, K (Decoding Rate)=22026.02 | 19.5 | — | — | |
| ConfidenceModel=TraDo-8B-Instruct, K (Decoding Rate)=22026.02 | 19.5 | — | — | |
| MarginModel=TraDo-8B-Instruct, K (Decoding Rate)=22026.02 | 18.9 | — | — |