Codebase generation on URL Shortener App (test)
91.6Feature Completeness (%)Code-L2MAC
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Code-L2MAC2023.10 | 91.6 | 0 | 330 | 14 | |
| GPT42023.10 | 53.6 | 0 | 119 | 2.56 | |
| CodeT2023.10 | 52.9 | 0.05 | 110 | 3.6 | |
| Self-Refine2023.10 | 47.9 | 0.05 | 124 | 3.65 | |
| Reflexion2023.10 | 38.8 | 0.1 | 96.2 | 2.35 | |
| AutoGPTBackbone=GPT-42023.10 | 25.3 | 0 | 136 | 3.3 |