Code Generation on MBPP (pass@1, pass@5, diversity, avg. KL)
60.65Pass@1Multiple Adaptations (Random)
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| Multiple Adaptations (Random)sampling temperature (τ)=02024.11 | 60.65 | 70.42 | 0.64 | 0.014 | |
| SPA (Influence)sampling temperature (τ)=02024.11 | 60.46 | 73.68 | 0.78 | 0.02 | |
| Multiple Adaptations (Lexical)sampling temperature (τ)=02024.11 | 60.33 | 71.17 | 0.71 | 0.018 | |
| Single Adaptationsampling temperature (τ)=0.12024.11 | 60.15 | 64.16 | 0.53 | — |