Bug Injection on HumanEval+
97.11ASRXOXO + GCGS
Evaluation results
| Method | Links | ||
|---|---|---|---|
| XOXO + GCGSModel=Llama 3.1 8B2025.03 | 97.11 | 65 | |
| XOXOModel=Llama 3.1 8B2025.03 | 93.73 | 90 | |
| XOXOModel=Claude 3.5 Sonnet v22025.03 | 92 | 145 | |
| XOXO + GCGSModel=DeepSeek Coder 6.7B2025.03 | 90.73 | 154 | |
| XOXOModel=DeepSeek Coder 6.7B2025.03 | 88.36 | 165 | |
| XOXO + GCGSModel=DeepSeek Coder 33B2025.03 | 85.69 | 240 | |
| XOXO + GCGSModel=GPT 4o2025.03 | 81.82 | 150 | |
| XOXO + GCGSModel=Codestral 22B2025.03 | 78.7 | 263 | |
| XOXOModel=DeepSeek Coder 33B2025.03 | 76.9 | 283 | |
| XOXO + GCGSModel=Qwen 2.5 Coder 7B2025.03 | 76.03 | 299 | |
| XOXOModel=Codestral 22B2025.03 | 74.15 | 273 | |
| XOXOModel=Qwen 2.5 Coder 7B2025.03 | 70.84 | 317 | |
| XOXO + GCGSModel=Qwen 2.5 Coder 32B2025.03 | 50.63 | 492 | |
| XOXOModel=Qwen 2.5 Coder 32B2025.03 | 43.5 | 501 |