Multi-task Code Intelligence (Overall) on CodefuseEval & CodeXGLUE
46Overall ScoreGALLa
Evaluation Results
| Method | Links | |
|---|---|---|
| GALLaBackbone=Qwen2.5-Coder 14B, Setting=G2C + GraphQA2024.09 | 46 | |
| GALLaBackbone=LLAMA3 8B, Setting=G2C + GraphQA2024.09 | 45.4 | |
| GALLaBackbone=LLAMA3 8B, Setting=G2C2024.09 | 45 | |
| LLAMA3 8BSetting=Baseline2024.09 | 44.6 | |
| GALLaBackbone=Qwen2.5-Coder 1.5B, Setting=G2C + GraphQA2024.09 | 44.2 | |
| GALLaBackbone=Qwen2.5-Coder 14B, Setting=G2C2024.09 | 44.2 | |
| GALLaBackbone=Qwen2.5-Coder 1.5B, Setting=G2C2024.09 | 44.1 | |
| GALLaBackbone=Phi-1 1.3B, Setting=G2C + GraphQA2024.09 | 43 | |
| Qwen2.5-Coder 14BSetting=Baseline2024.09 | 42.3 | |
| GALLaBackbone=Phi-1 1.3B, Setting=G2C2024.09 | 42.2 | |
| Phi-1 1.3BSetting=Baseline2024.09 | 41.4 | |
| Qwen2.5-Coder 1.5BSetting=Baseline2024.09 | 39.3 | |
| GALLaBackbone=CodeGen 350M, Setting=G2C + GraphQA2024.09 | 36.6 | |
| GALLaBackbone=CodeGen 350M, Setting=G2C2024.09 | 35.2 | |
| CodeGen 350MSetting=Baseline2024.09 | 34.7 | |
| GALLaBackbone=LLAMA2 7B, Setting=G2C2024.09 | 28.8 | |
| GALLaBackbone=LLAMA2 7B, Setting=G2C + GraphQA2024.09 | 27.4 | |
| LLAMA2 7BSetting=Baseline2024.09 | 26.9 | |
| GALLaBackbone=StarCoder 1B, Setting=G2C + GraphQA2024.09 | 18.9 | |
| GALLaBackbone=StarCoder 1B, Setting=G2C2024.09 | 16.7 | |
| StarCoder 1BSetting=Baseline2024.09 | 14 |