Code Generation on MBPP (Accuracy, Runtime, Tokens)
0.406Accuracyρ-EOS
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| ρ-EOSInitial Length=64, Threshold Configuration=Asym2026.01 | 0.406 | 335.9 | 594.4 | 0.555 | 2.03 | |
| ρ-EOSInitial Length=64, Threshold Configuration=Sym2026.01 | 0.396 | 324 | 507.4 | 0.639 | 2.031 | |
| DAEDALInitial Length=642026.01 | 0.394 | 308.4 | 558.3 | 0.552 | 3.551 | |
| Fixed-Length DenoisingSequence Length=20482026.01 | 0.388 | 336 | 2,048 | 0.163 | 15.172 | |
| Fixed-Length DenoisingSequence Length=5122026.01 | 0.387 | 331 | 512 | 0.645 | 1.728 | |
| Fixed-Length DenoisingSequence Length=10242026.01 | 0.375 | 335 | 1,024 | 0.329 | 4.76 | |
| Fixed-Length DenoisingSequence Length=2562026.01 | 0.367 | 232 | 256 | 0.904 | 0.713 | |
| Fixed-Length DenoisingSequence Length=1282026.01 | 0.288 | 122 | 128 | 0.955 | 0.332 | |
| Fixed-Length DenoisingSequence Length=642026.01 | 0.21 | 61 | 64 | 0.949 | 0.176 |