Programming on HumanEval
79.5ACC1ChatGPT o1-mini
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| ChatGPT o1-miniModel=o1-mini2024.12 | 79.5 | 4.3 | 14.8 | |
| ChatGPT 4oModel=4o2024.12 | 72.6 | 6.8 | 21.9 | |
| ChatGPT 3.5-turboModel=3.5-turbo2024.12 | 50.9 | 10.6 | 28.3 |
Loading the SOTA2 catalog…