Code Generation on CodeContest (Pass@1, Pass@2, Pass@5)
10.77Pass@1MPSC
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| MPSCSolutions=100, Specifications=50, Test cases=50, Temperature=0.8, Top P=0.95, Frequency Penalty=0, Presence Penalty=02023.09 | 10.77 | 12.64 | 13.84 | |
| GPT-4Temperature=0.8, Top P=0.95, Frequency Penalty=0, Presence Penalty=02023.09 | 6.1 | 8.28 | 11.72 | |
| GPT-3.5-TurboTemperature=0.8, Top P=0.95, Frequency Penalty=0, Presence Penalty=02023.09 | 2.57 | 4.22 | 7.16 |