Adversarial Code Compliance on C++
100Decoupling ProbabilityDeepSeek-v3.2
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DeepSeek-v3.2Model Category=Open Source (Large), Strategy=RPA^C2026.01 | 100 | 61.1 | |
| Gemma-3-27bModel Category=Open Source (Small), Strategy=RPA^C2026.01 | 97.8 | 47.1 | |
| Llama-3.1-8BModel Category=Open Source (Small), Strategy=RPA^C2026.01 | 97.8 | 43.7 | |
| Gemini-2.5-FlashModel Category=Proprietary, Strategy=RPA^C2026.01 | 96.7 | 58.1 | |
| Qwen3-235BModel Category=Open Source (Large), Strategy=RPA^C2026.01 | 94.6 | 38.6 | |
| GPT-5Model Category=Proprietary, Strategy=RPA^C2026.01 | 91 | 50.1 | |
| Llama-3.2-3BModel Category=Open Source (Small), Strategy=RPA^C2026.01 | 80.7 | 26.7 | |
| GPT-OSS-120BModel Category=Open Source (Large), Strategy=RPA^C2026.01 | 33.3 | 2.2 | |
| GPT-5-MiniModel Category=Proprietary, Strategy=RPA^C2026.01 | 33.3 | 0.6 |