Code Generation on HumanEval+ (Skill-Level Win/Tie Rates)
90Beginner Win-rateGPT-5.1 vs. GPT-4o
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| GPT-5.1 vs. GPT-4oPrompt Type=Personalized2026.04 | 90 | 1 | 90 | 2 | 85 | 1 | 85 | 2 | |
| GPT-5.1 vs. GPT-OSS-20BPrompt Type=Personalized2026.04 | 60 | 5 | 63 | 3 | 39 | 1 | 63 | 3 | |
| GPT-5.1 vs. GPT-4oPrompt Type=Original2026.04 | 13 | 1 | 22 | 3 | 68 | 3 | 92 | 1 | |
| GPT-5.1 vs. GPT-OSS-20BPrompt Type=Control2026.04 | 12 | 1 | 10 | 1 | 20 | 2 | 87 | 1 | |
| GPT-5.1 vs. GPT-4oPrompt Type=Control2026.04 | 11 | 1 | 12 | 4 | 66 | 3 | 92 | 1 | |
| GPT-5.1 vs. GPT-OSS-20BPrompt Type=Original2026.04 | 3 | 1 | 2 | 1 | 12 | 2 | 85 | 0 |