Safe Code Generation on C++ Safe Code Generation Dataset
97.18Safe Code RateClaude Sonnet 4.6
Evaluation results
| Method | Links | |
|---|---|---|
| Claude Sonnet 4.6Prompting Condition=+Comp. I (SPARK Component I)2026.06 | 97.18 | |
| Claude Haiku 4.5Prompting Condition=+Comp. I (SPARK Component I)2026.06 | 95.77 | |
| Claude Sonnet 4.6Prompting Condition=Baseline2026.06 | 94.37 | |
| GPT-5.4Prompting Condition=+Comp. I (SPARK Component I)2026.06 | 94.37 | |
| Claude Haiku 4.5Prompting Condition=Baseline2026.06 | 93.66 | |
| GPT-5.4-miniPrompting Condition=Baseline2026.06 | 90.14 | |
| GPT-5.4-miniPrompting Condition=+Comp. I (SPARK Component I)2026.06 | 89.44 | |
| GPT-5.4Prompting Condition=Baseline2026.06 | 89.44 | |
| DeepSeek V4 ProPrompting Condition=+Comp. I (SPARK Component I)2026.06 | 79.58 | |
| GPT-5.5Prompting Condition=+Comp. I (SPARK Component I)2026.06 | 66.2 | |
| GPT-5.5Prompting Condition=Baseline2026.06 | 64.79 | |
| DeepSeek V4 FlashPrompting Condition=+Comp. I (SPARK Component I)2026.06 | 59.86 | |
| DeepSeek V4 ProPrompting Condition=Baseline2026.06 | 59.15 | |
| DeepSeek V4 FlashPrompting Condition=Baseline2026.06 | 21.13 | |
| GPT-5.4Prompting Condition=Plain2026.06 | 7.04 | |
| Claude Haiku 4.5Prompting Condition=Plain2026.06 | 5.63 | |
| Claude Sonnet 4.6Prompting Condition=Plain2026.06 | 4.93 | |
| DeepSeek V4 ProPrompting Condition=Plain2026.06 | 4.93 | |
| GPT-5.5Prompting Condition=Plain2026.06 | 4.23 | |
| DeepSeek V4 FlashPrompting Condition=Plain2026.06 | 3.52 | |
| GPT-5.4-miniPrompting Condition=Plain2026.06 | 2.11 |