Code Repair on CodeXGLUE
15.7Repair RateGALLa
Evaluation Results
| Method | Links | |
|---|---|---|
| GALLaBackbone=Qwen2.5-Coder 1.5B, Setting=G2C2024.09 | 15.7 | |
| GALLaBackbone=Qwen2.5-Coder 1.5B, Setting=G2C + GraphQA2024.09 | 15 | |
| GALLaBackbone=Qwen2.5-Coder 14B, Setting=G2C + GraphQA2024.09 | 13.8 | |
| GALLaBackbone=LLAMA3 8B, Setting=G2C + GraphQA2024.09 | 13.6 | |
| GALLaBackbone=LLAMA3 8B, Setting=G2C2024.09 | 13.4 | |
| Qwen2.5-Coder 1.5BSetting=Baseline2024.09 | 13.2 | |
| GALLaBackbone=Qwen2.5-Coder 14B, Setting=G2C2024.09 | 13.1 | |
| Qwen2.5-Coder 14BSetting=Baseline2024.09 | 12.7 | |
| LLAMA3 8BSetting=Baseline2024.09 | 12.1 | |
| GALLaBackbone=Phi-1 1.3B, Setting=G2C2024.09 | 10.9 | |
| GALLaBackbone=Phi-1 1.3B, Setting=G2C + GraphQA2024.09 | 10.2 | |
| Phi-1 1.3BSetting=Baseline2024.09 | 10.1 | |
| CodeGen 350MSetting=Baseline2024.09 | 9 | |
| GALLaBackbone=CodeGen 350M, Setting=G2C + GraphQA2024.09 | 8.6 | |
| GALLaBackbone=CodeGen 350M, Setting=G2C2024.09 | 8.5 | |
| GALLaBackbone=LLAMA2 7B, Setting=G2C + GraphQA2024.09 | 1.6 | |
| GALLaBackbone=LLAMA2 7B, Setting=G2C2024.09 | 1 | |
| LLAMA2 7BSetting=Baseline2024.09 | 0.8 | |
| GALLaBackbone=StarCoder 1B, Setting=G2C2024.09 | 0.6 | |
| StarCoder 1BSetting=Baseline2024.09 | 0.4 | |
| GALLaBackbone=StarCoder 1B, Setting=G2C + GraphQA2024.09 | 0.3 |