Code Generation on RepoBench-P Python XF-First
52.4Exact Match (EM)Ours
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| OursBackbone=Llama-3.1-8B2026.05 | 52.4 | 73.8 | 118 | 3.8 | |
| RepoformerBackbone=Llama-3.1-8B2026.05 | 51.8 | 73.4 | 245 | 5.6 | |
| RepoHyperBackbone=Llama-3.1-8B2026.05 | 51.2 | 73.1 | 268 | 5.9 | |
| RepoCoderBackbone=Llama-3.1-8B2026.05 | 49.8 | 71.5 | 285 | 6.2 | |
| Sync-RAGBackbone=Llama-3.1-8B2026.05 | 48.3 | 70.2 | 312 | 6.8 | |
| No-RAGBackbone=Llama-3.1-8B2026.05 | 38.2 | 62.4 | 45 | 1.8 |