Safe Code Generation on Python Safe Code Generation Dataset
100Safe Code RateClaude Haiku 4.5
Evaluation Results
| Method | Links | |
|---|---|---|
| Claude Haiku 4.5Prompting Condition=+Comp. I (SPARK Component I)2026.06 | 100 | |
| Claude Sonnet 4.6Prompting Condition=+Comp. I (SPARK Component I)2026.06 | 98.59 | |
| GPT-5.4Prompting Condition=+Comp. I (SPARK Component I)2026.06 | 95.77 | |
| GPT-5.5Prompting Condition=+Comp. I (SPARK Component I)2026.06 | 94.96 | |
| Claude Haiku 4.5Prompting Condition=Baseline2026.06 | 91.55 | |
| Claude Sonnet 4.6Prompting Condition=Baseline2026.06 | 91.55 | |
| GPT-5.4Prompting Condition=Baseline2026.06 | 83.8 | |
| GPT-5.4-miniPrompting Condition=+Comp. I (SPARK Component I)2026.06 | 81.69 | |
| DeepSeek V4 ProPrompting Condition=+Comp. I (SPARK Component I)2026.06 | 80.28 | |
| GPT-5.5Prompting Condition=Baseline2026.06 | 80.14 | |
| DeepSeek V4 FlashPrompting Condition=+Comp. I (SPARK Component I)2026.06 | 71.83 | |
| GPT-5.4-miniPrompting Condition=Baseline2026.06 | 62.68 | |
| DeepSeek V4 ProPrompting Condition=Baseline2026.06 | 50 | |
| Claude Haiku 4.5Prompting Condition=Plain2026.06 | 34.51 | |
| GPT-5.4Prompting Condition=Plain2026.06 | 30.28 | |
| GPT-5.5Prompting Condition=Plain2026.06 | 26.62 | |
| DeepSeek V4 FlashPrompting Condition=Baseline2026.06 | 25.35 | |
| Claude Sonnet 4.6Prompting Condition=Plain2026.06 | 11.97 | |
| DeepSeek V4 ProPrompting Condition=Plain2026.06 | 7.75 | |
| GPT-5.4-miniPrompting Condition=Plain2026.06 | 7.04 | |
| DeepSeek V4 FlashPrompting Condition=Plain2026.06 | 5.63 |