Sabotage detection on BigCodeBench Sabotage (reasoning LLM attacker)
0.87log-AUROCExtract-and-Evaluate
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Extract-and-EvaluateMonitor Model=GPT-4.12026.01 | 0.87 | — | |
| Action-onlyMonitor Model=GPT-4.12026.01 | 0.73 | — | |
| Extract-and-EvaluateMonitor Model=Claude-3.5-Haiku2026.01 | 0.73 | — | |
| Action-onlyMonitor Model=Claude-3.5-Haiku2026.01 | 0.61 | — | |
| CoT-onlyMonitor Model=GPT-4.12026.01 | 0.58 | — | |
| CoT+actionMonitor Model=GPT-4.12026.01 | 0.55 | — | |
| CoT+actionMonitor Model=Claude-3.5-Haiku2026.01 | 0.48 | — | |
| CoT-onlyMonitor Model=Claude-3.5-Haiku2026.01 | 0.45 | — |