Adversarial Code Compliance on Python (Py)
99.4Decoupling ProbabilityGemma-3-27b
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Gemma-3-27bModel Category=Open Source (Small), Strategy=RPA^C2026.01 | 99.4 | 44.4 | |
| Llama-3.1-8BModel Category=Open Source (Small), Strategy=RPA^C2026.01 | 98.9 | 55.6 | |
| DeepSeek-v3.2Model Category=Open Source (Large), Strategy=RPA^C2026.01 | 95.5 | 48.7 | |
| Qwen3-235BModel Category=Open Source (Large), Strategy=RPA^C2026.01 | 88.7 | 26.8 | |
| Llama-3.2-3BModel Category=Open Source (Small), Strategy=RPA^C2026.01 | 82.2 | 55.6 | |
| Gemini-2.5-FlashModel Category=Proprietary, Strategy=RPA^C2026.01 | 79.8 | 39.3 | |
| GPT-5Model Category=Proprietary, Strategy=RPA^C2026.01 | 62.1 | 19.6 | |
| GPT-OSS-120BModel Category=Open Source (Large), Strategy=RPA^C2026.01 | 25.6 | 0.9 | |
| GPT-5-MiniModel Category=Proprietary, Strategy=RPA^C2026.01 | 23 | 0.1 |