Defect Detection on CodeXGLUE
0.534Defect RateLLAMA2 7B
Evaluation Results
| Method | Links | |
|---|---|---|
| LLAMA2 7BSetting=Baseline2024.09 | 0.534 | |
| GALLaBackbone=LLAMA2 7B, Setting=G2C2024.09 | 0.535 | |
| StarCoder 1BSetting=Baseline2024.09 | 0.542 | |
| GALLaBackbone=StarCoder 1B, Setting=G2C2024.09 | 0.548 | |
| GALLaBackbone=LLAMA2 7B, Setting=G2C + GraphQA2024.09 | 0.552 | |
| GALLaBackbone=CodeGen 350M, Setting=G2C2024.09 | 0.565 | |
| GALLaBackbone=StarCoder 1B, Setting=G2C + GraphQA2024.09 | 0.565 | |
| LLAMA3 8BSetting=Baseline2024.09 | 0.567 | |
| GALLaBackbone=LLAMA3 8B, Setting=G2C2024.09 | 0.567 | |
| CodeGen 350MSetting=Baseline2024.09 | 0.569 | |
| GALLaBackbone=LLAMA3 8B, Setting=G2C + GraphQA2024.09 | 0.571 | |
| GALLaBackbone=Qwen2.5-Coder 14B, Setting=G2C + GraphQA2024.09 | 0.58 | |
| Qwen2.5-Coder 1.5BSetting=Baseline2024.09 | 0.584 | |
| Qwen2.5-Coder 14BSetting=Baseline2024.09 | 0.585 | |
| GALLaBackbone=CodeGen 350M, Setting=G2C + GraphQA2024.09 | 0.587 | |
| GALLaBackbone=Qwen2.5-Coder 14B, Setting=G2C2024.09 | 0.591 | |
| GALLaBackbone=Phi-1 1.3B, Setting=G2C + GraphQA2024.09 | 0.603 | |
| GALLaBackbone=Qwen2.5-Coder 1.5B, Setting=G2C2024.09 | 0.603 | |
| GALLaBackbone=Phi-1 1.3B, Setting=G2C2024.09 | 0.604 | |
| Phi-1 1.3BSetting=Baseline2024.09 | 0.611 | |
| GALLaBackbone=Qwen2.5-Coder 1.5B, Setting=G2C + GraphQA2024.09 | 0.613 |