Adversarial Code Compliance on Java
91.7Decoupling ProbabilityLlama-3.1-8B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Llama-3.1-8BModel Category=Open Source (Small), Strategy=RPA^C2026.01 | 91.7 | 41.5 | |
| DeepSeek-v3.2Model Category=Open Source (Large), Strategy=RPA^C2026.01 | 90 | 48.4 | |
| Gemma-3-27bModel Category=Open Source (Small), Strategy=RPA^C2026.01 | 89.1 | 36.1 | |
| Qwen3-235BModel Category=Open Source (Large), Strategy=RPA^C2026.01 | 85 | 31.4 | |
| Llama-3.2-3BModel Category=Open Source (Small), Strategy=RPA^C2026.01 | 81.6 | 30.1 | |
| Gemini-2.5-FlashModel Category=Proprietary, Strategy=RPA^C2026.01 | 78 | 39 | |
| GPT-5Model Category=Proprietary, Strategy=RPA^C2026.01 | 60.1 | 17.2 | |
| GPT-5-MiniModel Category=Proprietary, Strategy=RPA^C2026.01 | 34.4 | 0.9 | |
| GPT-OSS-120BModel Category=Open Source (Large), Strategy=RPA^C2026.01 | 32.1 | 1.3 |