Long Code Completion on RepoBench 4k-8k
53.3Edit SimilarityHiROPE
Evaluation results
| Method | Links | |
|---|---|---|
| HiROPEBackbone=LLaMA-2 7B2024.03 | 53.3 | |
| ReROPEBackbone=LLaMA-2 7B2024.03 | 47.85 | |
| ReROPEBackbone=Vicuna 7B2024.03 | 43.65 | |
| HiROPEBackbone=TinyLLaMA 1.1B2024.03 | 43.53 | |
| HiROPEBackbone=Vicuna 7B2024.03 | 42.3 | |
| ReROPEBackbone=TinyLLaMA 1.1B2024.03 | 40.5 | |
| HiROPEBackbone=ShearedLLaMA 1.3B2024.03 | 39.98 | |
| ReROPEBackbone=ShearedLLaMA 1.3B2024.03 | 37.37 | |
| originBackbone=TinyLLaMA 1.1B2024.03 | 7.53 | |
| originBackbone=LLaMA-2 7B2024.03 | 6.79 | |
| originBackbone=ShearedLLaMA 1.3B2024.03 | 2.94 | |
| originBackbone=Vicuna 7B2024.03 | 2.49 |