Hack Detection on Terminal-Wrench (full traces)
96.79AUCgpt-5.4
Evaluation results
| Method | Links | ||
|---|---|---|---|
| gpt-5.4Condition=hack, Context=judges see hack prompt + reasoning, Judge Model=gpt-5.4, Positives=3632, Negatives=23522026.06 | 96.79 | 82.35 | |
| gpt-5.4Condition=sanitized, Context=judges see reasoning, prompt removed, Judge Model=gpt-5.4, Positives=3632, Negatives=23522026.06 | 94.74 | 61.87 | |
| gpt-5.4Condition=stripped, Context=action surface only, Judge Model=gpt-5.4, Positives=3632, Negatives=23522026.06 | 91.68 | 44 |