Repository-level code-completion on RepoBench (test)
65.9Exact-match AccuracyRLCoder
Evaluation results
| Method | Links | |
|---|---|---|
| RLCoderParams=1.1B, Latency=8.8s, Offline=50.4s2026.02 | 65.9 | |
| GraphCoderParams=16B, Latency=17.5s, Offline=10.2s2026.02 | 64.1 | |
| HEF (end-to-end)Params=1.8B, Latency=0.68s, Offline=35.1s2026.02 | 61.3 | |
| DRACOParams=7.1B, Latency=11.0s, Offline=14.3s2026.02 | 60.3 | |
| HEF (separate)Params=1.8B, Latency=0.94s, Offline=35.1s2026.02 | 56.9 | |
| Qwen-2.5-Coder-1.3BParams=1.5B, Latency=0.50s2026.02 | 49.1 | |
| RepoFusionParams=0.22B, Latency=0.9s, Offline=24.6s2026.02 | 39.8 |