Code Generation on HumanEval (Accuracy, Speedup)
55.5AccuracyToken Order
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Token OrderTokens per step=12026.04 | 55.5 | 1.3 | |
| KLASS2026.04 | 55.5 | 1.1 | |
| DEMASK2026.04 | 54.9 | 1.7 | |
| EntropyTokens per step=12026.04 | 53.7 | 1 | |
| Top-1Tokens per step=12026.04 | 53.7 | 1 | |
| EntropyTokens per step=22026.04 | 31.1 | 1.1 | |
| Top-1Tokens per step=22026.04 | 31.1 | 1.1 | |
| Token OrderTokens per step=22026.04 | 24.4 | 1.1 |