Code Generation on CodeProjectEval (Per-project)
140Pass CountCo-Coder
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Co-CoderProject=pyjwt2026.05 | 140 | 1,310 | 0.5 | |
| File-based ParallelProject=pyjwt2026.05 | 126 | 1,215 | 1.48 | |
| Co-CoderProject=tinydb2026.05 | 125.7 | 656 | 0.3 | |
| SequentialProject=tinydb2026.05 | 110 | 2,067 | 0.6 | |
| SequentialProject=pyjwt2026.05 | 79 | 2,222 | 0.79 | |
| Co-CoderProject=cookiecutter2026.05 | 74.7 | 1,693 | 0.87 | |
| Co-CoderProject=bplustree2026.05 | 73 | 1,311 | 0.56 | |
| Claude Code w/ Agent TeamsProject=tinydb2026.05 | 67.3 | 301 | 0.23 | |
| Co-CoderProject=imapclient2026.05 | 63.3 | 1,104 | 0.54 | |
| File-based ParallelProject=tinydb2026.05 | 61.7 | 696 | 0.85 | |
| SequentialProject=cookiecutter2026.05 | 59 | 4,260 | 1.8 | |
| Co-CoderProject=simpy2026.05 | 58 | 1,710 | 0.68 | |
| Claude Code w/ Agent TeamsProject=bplustree2026.05 | 55.7 | 1,055 | 0.67 | |
| File-based ParallelProject=imapclient2026.05 | 49 | 1,330 | 1.78 | |
| File-based ParallelProject=cookiecutter2026.05 | 46.7 | 2,180 | 2.37 | |
| SequentialProject=imapclient2026.05 | 46.3 | 3,482 | 1.13 | |
| Co-CoderProject=drf-simplejwt2026.05 | 45.3 | 1,238 | 0.82 | |
| File-based ParallelProject=simpy2026.05 | 39.3 | 3,472 | 1.89 | |
| Co-CoderProject=Avg2026.05 | 34.1 | 1,315 | 0.67 | |
| Claude Code w/ Agent TeamsProject=pyjwt2026.05 | 29 | 156 | 0.21 | |
| Claude Code w/ Agent TeamsProject=simpy2026.05 | 24 | 1,081 | 0.69 | |
| File-based ParallelProject=Avg2026.05 | 23.3 | 1,771 | 1.65 | |
| File-based ParallelProject=bplustree2026.05 | 22 | 2,312 | 1.3 | |
| SequentialProject=Avg2026.05 | 20.1 | 2,756 | 1.03 | |
| SequentialProject=bplustree2026.05 | 18 | 2,192 | 0.99 | |
| Claude Code w/ Agent TeamsProject=csvs-to-sqlite2026.05 | 16.7 | 1,746 | 0.51 | |
| Claude Code w/ Agent TeamsProject=zxcvbn2026.05 | 16.3 | 306 | 0.14 | |
| Co-CoderProject=zxcvbn2026.05 | 16.3 | 823 | 0.5 | |
| Claude Code w/ Agent TeamsProject=Avg2026.05 | 16.3 | 680 | 0.38 | |
| Claude Code w/ Agent TeamsProject=cookiecutter2026.05 | 15.7 | 338 | 0.21 | |
| Co-CoderProject=csvs-to-sqlite2026.05 | 15.7 | 1,020 | 0.25 | |
| File-based ParallelProject=zxcvbn2026.05 | 14 | 854 | 0.87 | |
| SequentialProject=zxcvbn2026.05 | 13 | 1,479 | 0.66 | |
| File-based ParallelProject=csvs-to-sqlite2026.05 | 10.7 | 1,739 | 0.68 | |
| SequentialProject=simpy2026.05 | 10.3 | 3,147 | 1.42 | |
| SequentialProject=drf-simplejwt2026.05 | 6.7 | 3,633 | 1.42 | |
| SequentialProject=csvs-to-sqlite2026.05 | 5 | 1,525 | 0.46 | |
| File-based ParallelProject=drf-simplejwt2026.05 | 4.7 | 2,022 | 1.97 | |
| Claude Code w/ Agent TeamsProject=imapclient2026.05 | 3.3 | 261 | 0.2 | |
| Claude Code w/ Agent TeamsProject=drf-simplejwt2026.05 | 0 | 128 | 0.02 |