Adversarial Code Compliance on C
100Decoupling ProbabilityLlama-3.1-8B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Llama-3.1-8BModel Category=Open Source (Small), Strategy=RPA^C2026.01 | 100 | 52.7 | |
| DeepSeek-v3.2Model Category=Open Source (Large), Strategy=RPA^C2026.01 | 97.5 | 45.9 | |
| Qwen3-235BModel Category=Open Source (Large), Strategy=RPA^C2026.01 | 87.6 | 24.7 | |
| Gemma-3-27bModel Category=Open Source (Small), Strategy=RPA^C2026.01 | 82.6 | 24.3 | |
| Gemini-2.5-FlashModel Category=Proprietary, Strategy=RPA^C2026.01 | 79.8 | 35.2 | |
| GPT-5Model Category=Proprietary, Strategy=RPA^C2026.01 | 73.2 | 26.5 | |
| Llama-3.2-3BModel Category=Open Source (Small), Strategy=RPA^C2026.01 | 66.5 | 14.3 | |
| GPT-OSS-120BModel Category=Open Source (Large), Strategy=RPA^C2026.01 | 38.5 | 2 | |
| GPT-5-MiniModel Category=Proprietary, Strategy=RPA^C2026.01 | 27.5 | 0.6 |