Code Generation on HumanEval (Accuracy, Token Efficiency, and Runtime)
48.2AccFixed-Length Denoising
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Fixed-Length DenoisingSequence Length=20482026.01 | 48.2 | 691.1 | 2,048 | 33.8 | 16,046 | |
| Fixed-Length DenoisingSequence Length=5122026.01 | 45.1 | 462 | 512 | 90.2 | 1,474 | |
| Fixed-Length DenoisingSequence Length=10242026.01 | 45.1 | 631.1 | 1,024 | 61.6 | 4,569 | |
| DAEDALInitial Length=642026.01 | 44.5 | 494.7 | 813 | 64.8 | 1,283 | |
| ρ-EOSInitial Length=64, Threshold Configuration=Asym2026.01 | 43.9 | 643.1 | 769.9 | 83.5 | 593 | |
| ρ-EOSInitial Length=64, Threshold Configuration=Sym2026.01 | 43.3 | 570.4 | 654 | 88.4 | 580 | |
| Fixed-Length DenoisingSequence Length=2562026.01 | 36.6 | 247.3 | 256 | 96.6 | 543 | |
| Fixed-Length DenoisingSequence Length=1282026.01 | 26.2 | 125 | 128 | 97.7 | 230 | |
| Fixed-Length DenoisingSequence Length=642026.01 | 17.1 | 59.6 | 64 | 93.2 | 111 |