Case-level safety detection on ImpossibleBench
100ROC AUCMonitor
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| MonitorTPC=100, Model=Qwen3.52026.04 | 100 | — | |
| BayesianTPC=100, Model=Qwen3.52026.04 | 100 | — | |
| BayesianTPC=100, Model=GPT-5.4m2026.04 | 100 | — | |
| BufferTPC=100, Model=Qwen3.52026.04 | 99.4 | — | |
| BufferTPC=100, Model=GPT-5.4m2026.04 | 99.4 | — | |
| MeerkatTPC=100, Model=Qwen3.52026.04 | 98 | -0.02 | |
| BufferTPC=50, Model=Qwen3.52026.04 | 97.1 | — | |
| BufferTPC=50, Model=GPT-5.4m2026.04 | 97.1 | — | |
| BayesianTPC=50, Model=Qwen3.52026.04 | 96 | — | |
| BayesianTPC=50, Model=GPT-5.4m2026.04 | 96 | — | |
| MeerkatTPC=50, Model=Qwen3.52026.04 | 95.1 | -0.02 | |
| MonitorTPC=50, Model=Qwen3.52026.04 | 94 | — | |
| Naive AgentTPC=100, Model=GPT-5.4m2026.04 | 92.9 | — | |
| MeerkatTPC=10, Model=Qwen3.52026.04 | 92 | 0.02 | |
| MonitorTPC=100, Model=GPT-5.4m2026.04 | 90.1 | — | |
| BufferTPC=10, Model=Qwen3.52026.04 | 90 | — | |
| BufferTPC=10, Model=GPT-5.4m2026.04 | 90 | — | |
| MeerkatTPC=100, Model=GPT-5.4m2026.04 | 90 | -0.1 | |
| BayesianTPC=10, Model=Qwen3.52026.04 | 88.3 | — | |
| BayesianTPC=10, Model=GPT-5.4m2026.04 | 88.3 | — | |
| MeerkatTPC=25, Model=Qwen3.52026.04 | 85.8 | 0.015 | |
| Naive AgentTPC=50, Model=Qwen3.52026.04 | 85.1 | — | |
| BufferTPC=25, Model=Qwen3.52026.04 | 84.3 | — | |
| BufferTPC=25, Model=GPT-5.4m2026.04 | 84.3 | — | |
| MeerkatTPC=10, Model=GPT-5.4m2026.04 | 83 | -0.07 | |
| BayesianTPC=25, Model=GPT-5.4m2026.04 | 82.7 | — | |
| BayesianTPC=25, Model=Qwen3.52026.04 | 82.6 | — | |
| MeerkatTPC=50, Model=GPT-5.4m2026.04 | 81.1 | -0.16 | |
| MonitorTPC=50, Model=GPT-5.4m2026.04 | 80.4 | — | |
| MonitorTPC=10, Model=Qwen3.52026.04 | 80 | — | |
| Naive AgentTPC=100, Model=Qwen3.52026.04 | 79.8 | — | |
| Naive AgentTPC=10, Model=Qwen3.52026.04 | 79.4 | — | |
| MeerkatTPC=25, Model=GPT-5.4m2026.04 | 78.2 | -0.061 | |
| MonitorTPC=25, Model=Qwen3.52026.04 | 78 | — | |
| MonitorTPC=10, Model=GPT-5.4m2026.04 | 77.2 | — | |
| MonitorTPC=25, Model=GPT-5.4m2026.04 | 74 | — | |
| Naive AgentTPC=25, Model=Qwen3.52026.04 | 64.9 | — | |
| Naive AgentTPC=50, Model=GPT-5.4m2026.04 | 59.8 | — | |
| Naive AgentTPC=10, Model=GPT-5.4m2026.04 | 50 | — | |
| Naive AgentTPC=25, Model=GPT-5.4m2026.04 | 50 | — |