Code Generation on MBPP (Acc. %, Steps, Speed)
58AccuracyCOVER
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| COVERBackbone=Dream-Ins-7B, Length=2562026.02 | 58 | 42.35 | 4.36 | |
| COVERBackbone=Dream-Ins-7B, Length=5122026.02 | 57.6 | 35.84 | 11.64 | |
| baselineBackbone=Dream-Ins-7B, Length=5122026.02 | 57.2 | 512 | 1 | |
| SaberBackbone=Dream-Ins-7B, Length=2562026.02 | 57 | 207.17 | 1.31 | |
| WINOBackbone=Dream-Ins-7B, Length=2562026.02 | 57 | 57.08 | 4.23 | |
| baselineBackbone=Dream-Ins-7B, Length=2562026.02 | 56.8 | 256 | 1 | |
| WINOBackbone=Dream-Ins-7B, Length=5122026.02 | 56.2 | 62.11 | 7.82 | |
| SaberBackbone=Dream-Ins-7B, Length=5122026.02 | 55.4 | 461.57 | 1.22 | |
| COVERBackbone=LLaDA-Base-8B, Length=2562026.02 | 42.4 | 75.37 | 2.42 | |
| WINOBackbone=LLaDA-Base-8B, Length=2562026.02 | 41.8 | 82.08 | 1.55 | |
| SaberBackbone=LLaDA-Base-8B, Length=2562026.02 | 41.2 | 107.11 | 2.34 | |
| COVERBackbone=LLaDA-Base-8B, Length=5122026.02 | 41 | 94.03 | 3.82 | |
| baselineBackbone=LLaDA-Base-8B, Length=2562026.02 | 40.8 | 256 | 1 | |
| WINOBackbone=LLaDA-Base-8B, Length=5122026.02 | 40.4 | 125.35 | 1.92 | |
| SaberBackbone=LLaDA-Base-8B, Length=5122026.02 | 40.2 | 167.99 | 3.05 | |
| baselineBackbone=LLaDA-Base-8B, Length=5122026.02 | 39.8 | 512 | 1 | |
| COVERBackbone=LLaDA-1.5-8B, Length=5122026.02 | 39.4 | 123.48 | 2.83 | |
| COVERBackbone=LLaDA-Ins-8B, Length=2562026.02 | 39 | 69.75 | 2.35 | |
| COVERBackbone=LLaDA-Ins-8B, Length=5122026.02 | 38.8 | 115.24 | 3.08 | |
| baselineBackbone=LLaDA-1.5-8B, Length=2562026.02 | 38.6 | 256 | 1 | |
| WINOBackbone=LLaDA-Ins-8B, Length=5122026.02 | 38.4 | 131.29 | 1.94 | |
| SaberBackbone=LLaDA-1.5-8B, Length=5122026.02 | 38.4 | 185.5 | 2.7 | |
| baselineBackbone=LLaDA-1.5-8B, Length=5122026.02 | 38.2 | 512 | 1 | |
| WINOBackbone=LLaDA-1.5-8B, Length=5122026.02 | 38 | 139.51 | 1.84 | |
| WINOBackbone=LLaDA-1.5-8B, Length=2562026.02 | 37.8 | 89.99 | 1.65 | |
| COVERBackbone=LLaDA-1.5-8B, Length=2562026.02 | 37.8 | 70.52 | 2.45 | |
| SaberBackbone=LLaDA-Ins-8B, Length=2562026.02 | 37.6 | 114.86 | 2.25 | |
| baselineBackbone=LLaDA-Ins-8B, Length=5122026.02 | 37.6 | 512 | 1 | |
| SaberBackbone=LLaDA-1.5-8B, Length=2562026.02 | 37.6 | 124.42 | 2.38 | |
| SaberBackbone=LLaDA-Ins-8B, Length=5122026.02 | 37.4 | 168.44 | 2.96 | |
| baselineBackbone=LLaDA-Ins-8B, Length=2562026.02 | 37.2 | 256 | 1 | |
| WINOBackbone=LLaDA-Ins-8B, Length=2562026.02 | 36.2 | 87 | 1.64 |