Code Safety Evaluation on BigCodeBench 1.0 (test)
99.9Safety ScoreSelf-Rec (No Elicitation)
Evaluation results
| Method | Links | |
|---|---|---|
| Self-Rec (No Elicitation)Signal Jamming=None, Monitoring=UM+TM2026.02 | 99.9 | |
| Self-Rec (No Elicitation)Signal Jamming=Minifier, Monitoring=UM+TM2026.02 | 99.9 | |
| Self-Rec (No Elicitation)Signal Jamming=Summariser, Monitoring=UM+TM2026.02 | 99.9 | |
| Vague AcausalSignal Jamming=None, Monitoring=UM+TM2026.02 | 99.9 | |
| Vague AcausalSignal Jamming=Summariser, Monitoring=UM+TM2026.02 | 99.9 | |
| Causal (For/While)Signal Jamming=Summariser, Monitoring=UM+TM2026.02 | 99.9 | |
| Self-Rec (SFT)Signal Jamming=Minifier, Monitoring=UM+TM2026.02 | 99.6 | |
| Causal (For/While)Signal Jamming=None, Monitoring=UM+TM2026.02 | 99.6 | |
| Causal (For/While)Signal Jamming=Minifier, Monitoring=UM+TM2026.02 | 99.6 | |
| Vague AcausalSignal Jamming=Minifier, Monitoring=UM+TM2026.02 | 99.5 | |
| Self-Rec (SFT)Signal Jamming=None, Monitoring=UM+TM2026.02 | 99.3 | |
| Self-Rec (SFT)Signal Jamming=Summariser, Monitoring=UM+TM2026.02 | 99.2 | |
| Self-Rec (No Elicitation)Signal Jamming=None, Monitoring=UM-only2026.02 | 97.7 | |
| Self-Rec (No Elicitation)Signal Jamming=Minifier, Monitoring=UM-only2026.02 | 97.6 | |
| Self-Rec (No Elicitation)Signal Jamming=Summariser, Monitoring=UM-only2026.02 | 96.2 | |
| Vague AcausalSignal Jamming=Minifier, Monitoring=UM-only2026.02 | 95 | |
| Causal (For/While)Signal Jamming=Summariser, Monitoring=UM-only2026.02 | 91.7 | |
| Vague AcausalSignal Jamming=Summariser, Monitoring=UM-only2026.02 | 87.8 | |
| Vague AcausalSignal Jamming=None, Monitoring=UM-only2026.02 | 73.2 | |
| Self-Rec (SFT)Signal Jamming=Summariser, Monitoring=UM-only2026.02 | 73 | |
| Self-Rec (SFT)Signal Jamming=Minifier, Monitoring=UM-only2026.02 | 68.6 | |
| Causal (For/While)Signal Jamming=Minifier, Monitoring=UM-only2026.02 | 13.3 | |
| Self-Rec (SFT)Signal Jamming=None, Monitoring=UM-only2026.02 | 11.1 | |
| Causal (For/While)Signal Jamming=None, Monitoring=UM-only2026.02 | 6.7 |