Code Completion on RepoBench-P LongBench
52.05Pass@1Baseline
Evaluation results
| Method | Links | |
|---|---|---|
| BaselineModel=Llama 2-7B2026.02 | 52.05 | |
| KIVISinkModel=Llama 2-7B2026.02 | 52.04 | |
| InnerQBaseModel=Llama 2-7B2026.02 | 52.03 | |
| KIVIModel=Llama 2-7B2026.02 | 51.57 | |
| InnerQHybridModel=Llama 2-7B2026.02 | 50.43 | |
| BaselineModel=Llama 2-13B2026.02 | 49.78 | |
| InnerQBaseModel=Llama 2-13B2026.02 | 49.71 | |
| InnerQHybridModel=Llama 2-13B2026.02 | 49.05 | |
| KIVISinkModel=Llama 2-13B2026.02 | 49.03 | |
| KIVIModel=Llama 2-13B2026.02 | 48.59 | |
| InnerQSmallModel=Llama 2-13B2026.02 | 46.14 | |
| KIVISinkModel=Llama 3.1-8B2026.02 | 38.37 | |
| BaselineModel=Llama 3.1-8B2026.02 | 38.26 | |
| InnerQBaseModel=Llama 3.1-8B2026.02 | 38.23 | |
| KIVIModel=Llama 3.1-8B2026.02 | 38.21 | |
| InnerQHybridModel=Llama 3.1-8B2026.02 | 38.08 | |
| InnerQSmallModel=Llama 3.1-8B2026.02 | 37.93 | |
| BaselineModel=Llama 3.2-3B2026.02 | 29.56 | |
| KIVISinkModel=Llama 3.2-3B2026.02 | 29.44 | |
| KIVIModel=Llama 3.2-3B2026.02 | 29.36 | |
| InnerQSmallModel=Llama 3.2-3B2026.02 | 29.34 | |
| InnerQBaseModel=Llama 3.2-3B2026.02 | 29.19 | |
| InnerQHybridModel=Llama 3.2-3B2026.02 | 29.02 | |
| InnerQSmallModel=Llama 2-7B2026.02 | 28.71 | |
| KIVIModel=Llama 3.2-1B2026.02 | 17.33 | |
| BaselineModel=Llama 3.2-1B2026.02 | 16.61 | |
| InnerQSmallModel=Llama 3.2-1B2026.02 | 15.84 | |
| KIVISinkModel=Llama 3.2-1B2026.02 | 15.41 | |
| InnerQBaseModel=Llama 3.2-1B2026.02 | 14.54 | |
| InnerQHybridModel=Llama 3.2-1B2026.02 | 14.43 |