Code Generation on HumanEval (Acc (%), Step)
59.35Accuracy (%)KL < 0.001
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| KL < 0.001Sampler=KL-based, KL_threshold=0.0012025.11 | 59.35 | 73.94 | |
| KLASSSampler=KLASS2025.11 | 59.35 | 74.88 | |
| Top-1Sampler=Top-12025.11 | 58.53 | 256 | |
| conf > 0.9Sampler=confidence-based, confidence_threshold=0.92025.11 | 50 | 52.47 | |
| Top-2Sampler=Top-22025.11 | 42.88 | 128 | |
| RandomSampler=Random2025.11 | 18.09 | 256 |