Code Generation on MBPP (Pass@1, Pass@2, Pass@5)
72.78Pass@1MPSC
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| MPSCSolutions=100, Specifications=50, Test cases=50, Temperature=0.8, Top P=0.95, Frequency Penalty=0, Presence Penalty=02023.09 | 72.78 | 74.24 | 78.25 | |
| GPT-4Temperature=0.8, Top P=0.95, Frequency Penalty=0, Presence Penalty=02023.09 | 71.26 | 74.27 | 76.99 | |
| GPT-3.5-TurboTemperature=0.8, Top P=0.95, Frequency Penalty=0, Presence Penalty=02023.09 | 66.8 | 72.34 | 76.6 |