Code Generation on HumanEval (Acc, Steps, Speed)
54.9AccuracyASRD
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ASRDModel=Dream-Base-7B, Sequence Length=5122026.06 | 54.9 | 195.1 | 2.4 | |
| ASRDModel=Dream-Base-7B, Sequence Length=2562026.06 | 54.3 | 111.4 | 2 | |
| BaselineModel=Dream-Base-7B, Sequence Length=5122026.06 | 54.3 | 494.2 | 1 | |
| BaselineModel=Dream-Base-7B, Sequence Length=2562026.06 | 50.6 | 238.4 | 1 | |
| ASRDModel=LLaDA-1.5-8B, Sequence Length=5122026.06 | 47.6 | 107.4 | 4.1 | |
| BaselineModel=LLaDA-1.5-8B, Sequence Length=5122026.06 | 45.1 | 493.5 | 1 | |
| ASRDModel=LLaDA-1.5-8B, Sequence Length=2562026.06 | 42.1 | 72.3 | 3.2 | |
| BaselineModel=LLaDA-1.5-8B, Sequence Length=2562026.06 | 38.4 | 239.7 | 1 | |
| ASRDModel=LLaDA-Base-8B, Sequence Length=2562026.06 | 34.8 | 124.7 | 1.8 | |
| ASRDModel=LLaDA-Base-8B, Sequence Length=5122026.06 | 34.8 | 179.2 | 2.6 | |
| BaselineModel=LLaDA-Base-8B, Sequence Length=5122026.06 | 32.9 | 495.1 | 1 | |
| BaselineModel=LLaDA-Base-8B, Sequence Length=2562026.06 | 32.3 | 240.5 | 1 |