Python Code Generation on MBPP
54.5Normal (%)zephyr-gemma-7b
Evaluation results
| Method | Links | ||
|---|---|---|---|
| zephyr-gemma-7bDecoding=Greedy, Evaluation Protocol=Pass@1, Chain-of-thought=false, Few-shots=false2024.05 | 54.5 | 44.7 | |
| RPODecoding=Greedy, Evaluation Protocol=Pass@1, Chain-of-thought=false, Few-shots=false, Backbone=Gemma2024.05 | 54.2 | 46.3 | |
| DPODecoding=Greedy, Evaluation Protocol=Pass@1, Chain-of-thought=false, Few-shots=false, Backbone=Gemma2024.05 | 54.2 | 43.9 | |
| zephyr-7b-gemma-sftDecoding=Greedy, Evaluation Protocol=Pass@1, Chain-of-thought=false, Few-shots=false, Backbone=Gemma2024.05 | 50.3 | 44.2 |