Malicious Attack on Terminal Bench (held-out)
41.4Trajectory-level ASRHinted Hacker
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Hinted HackerDefense State=Original, Filtering=LLM Judge-filtered2026.06 | 41.4 | 100 | — | — | |
| Hinted HackerDefense State=Hardened, Filtering=LLM Judge-filtered, Method=hacker-fixer loop2026.06 | 34.9 | 87 | -6.5 | 0.026 | |
| Unhinted HackerDefense State=Original, Filtering=LLM Judge-filtered2026.06 | 31.5 | 46.8 | — | — | |
| Unhinted HackerDefense State=Hardened, Filtering=LLM Judge-filtered, Method=hacker-fixer loop2026.06 | 12.7 | 21.1 | -18.8 | 2 |