Code Generation on HumanEval (Accuracy, Rank)
20AccuracyTop-k Sampling
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Top-k SamplingModel Family=Gemma2026.05 | 20 | 2.63 | |
| Top-k SamplingModel Family=Granite2026.05 | 20 | 4.75 | |
| Beam Search (3)Model Family=Gemma2026.05 | 19 | 2.33 | |
| EDENModel Family=Gemma2026.05 | 19 | 2 | |
| Beam Search (3)Model Family=Granite2026.05 | 19 | 2.5 | |
| EDENModel Family=Granite2026.05 | 19 | 1.37 | |
| GreedyModel Family=Gemma2026.05 | 18 | 4.25 | |
| Top-p SamplingModel Family=Gemma2026.05 | 18 | 4.25 | |
| Min-p SamplingModel Family=Gemma2026.05 | 18 | 4.25 | |
| GreedyModel Family=Granite2026.05 | 18 | 4.25 | |
| Top-p SamplingModel Family=Granite2026.05 | 18 | 4.5 | |
| Min-p SamplingModel Family=Granite2026.05 | 18 | 3.62 |