List your company
ResearchFormerly WizWand

HumanEvalFix

Benchmarks

TaskDatasetSOTA resultTrendResults
Code Repair
HumanEvalFix (test)
59.1Success Rate (Python)
19
Bug-fixing
HumanEvalFix held-out 25% (eval)
5.2Utility Difference (Δπ)
3