Bug injection on MBPP+
99.89Attack Success Rate (ASR)XOXO + GCGS
Evaluation results
| Method | Links | ||
|---|---|---|---|
| XOXO + GCGSModel=DeepSeek Coder 6.7B2025.03 | 99.89 | 20 | |
| XOXOModel=Llama 3.1 8B2025.03 | 99.88 | 22 | |
| XOXO + GCGSModel=Llama 3.1 8B2025.03 | 99.88 | 22 | |
| XOXOModel=DeepSeek Coder 6.7B2025.03 | 99.55 | 25 | |
| XOXO + GCGSModel=Codestral 22B2025.03 | 99.36 | 37 | |
| XOXOModel=Codestral 22B2025.03 | 98.99 | 43 | |
| XOXOModel=Claude 3.5 Sonnet v22025.03 | 98.42 | 75 | |
| XOXO + GCGSModel=DeepSeek Coder 33B2025.03 | 96.41 | 80 | |
| XOXOModel=DeepSeek Coder 33B2025.03 | 95.27 | 84 | |
| XOXO + GCGSModel=Qwen 2.5 Coder 7B2025.03 | 84.53 | 169 | |
| XOXOModel=Qwen 2.5 Coder 7B2025.03 | 81.29 | 180 | |
| XOXO + GCGSModel=Qwen 2.5 Coder 32B2025.03 | 75.37 | 235 | |
| XOXOModel=Qwen 2.5 Coder 32B2025.03 | 73.17 | 228 | |
| XOXO + GCGSModel=GPT 4o2025.03 | 40.69 | 233 |