Secure Code Generation on Secure Code Generation Scenarios 1.0 (test)
0.971Security Success Rategemini-2.5-pro (Reflex)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| gemini-2.5-pro (Reflex)Model=gemini-2.5-pro, Configuration=Base+Reflex2025.12 | 0.971 | 0.949 | 23.7 | 24.3 | 1.3 | |
| gpt-3.5-turbo (Reflex)Model=gpt-3.5-turbo, Configuration=Base+Reflex2025.12 | 0.966 | 0.924 | 23.1 | 24.1 | 1.9 | |
| gpt-4o (Reflex)Model=gpt-4o, Configuration=Base+Reflex2025.12 | 0.95 | 0.949 | 23.7 | 23.8 | 1.3 | |
| qwen3-coder-plus (Reflex)Model=qwen3-coder-plus, Configuration=Base+Reflex2025.12 | 0.949 | 0.801 | 20.2 | 23.7 | 4.8 | |
| gpt-3.5-turboModel=gpt-3.5-turbo, Configuration=Base2025.12 | 0.937 | 0.88 | 22 | 23.4 | 3 | |
| gemini-2.5-proModel=gemini-2.5-pro, Configuration=Base2025.12 | 0.88 | 0.914 | 22.9 | 22 | 2.1 | |
| gpt-4oModel=gpt-4o, Configuration=Base2025.12 | 0.857 | 0.952 | 23.8 | 21.4 | 1.2 | |
| qwen3-coder-plusModel=qwen3-coder-plus, Configuration=Base2025.12 | 0.837 | 0.867 | 21.6 | 17.9 | 3.3 |