Long Code Completion on RepoBench 0-4k
52.82Edit SimilarityReROPE
Evaluation results
| Method | Links | |
|---|---|---|
| ReROPEBackbone=LLaMA-2 7B2024.03 | 52.82 | |
| HiROPEBackbone=LLaMA-2 7B2024.03 | 52.2 | |
| ReROPEBackbone=Vicuna 7B2024.03 | 42.78 | |
| HiROPEBackbone=TinyLLaMA 1.1B2024.03 | 42.48 | |
| HiROPEBackbone=ShearedLLaMA 1.3B2024.03 | 40.17 | |
| ReROPEBackbone=TinyLLaMA 1.1B2024.03 | 37.72 | |
| HiROPEBackbone=Vicuna 7B2024.03 | 37.1 | |
| ReROPEBackbone=ShearedLLaMA 1.3B2024.03 | 34.03 | |
| originBackbone=LLaMA-2 7B2024.03 | 8.29 | |
| originBackbone=TinyLLaMA 1.1B2024.03 | 7.91 | |
| originBackbone=ShearedLLaMA 1.3B2024.03 | 4.39 | |
| originBackbone=Vicuna 7B2024.03 | 3.67 |