GPU code optimization on GPU code optimization demonstration problems (test)
0.19Cost per IterationAUTO
Evaluation results
| Method | Links | ||
|---|---|---|---|
| AUTOLLM Provider=GPT-5, Number of Iterations=782025.11 | 0.19 | 14.82 | |
| AUTOLLM Provider=Sonnet-4.5, Number of Iterations=782025.11 | 0.34 | 26.52 | |
| AUTOLLM Provider=Opus-4.1, Number of Iterations=782025.11 | 1.71 | 133.38 | |
| AUTOLLM Provider=o3-pro, Number of Iterations=782025.11 | 2.04 | 159.12 | |
| Human DeveloperLevel=Entry-level, Hourly Rate=$38.39/hr, Number of Iterations=152025.11 | 29 | 435 | |
| Human DeveloperLevel=Median, Hourly Rate=$63.98/hr, Number of Iterations=102025.11 | 48 | 480 |