Misalignment monitoring on Misalignment Monitoring Dataset (test)
97AUROC (Deception)LLM Judge (Haiku, ind)
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| LLM Judge (Haiku, ind)Cost=1.00×2026.06 | 97 | 88.5 | 93.9 | 100 | 97.8 | 95.2 | 85.7 | 0.93 | |
| Probes + Haiku 4.5Cost=0.06×2026.06 | 94.6 | 90.1 | 86.6 | 100 | 97.2 | 95 | 87 | 0.47 | |
| LLM Judge (Qwen3, ind)Cost=0.09×2026.06 | 93.4 | 83 | 90.9 | 98.9 | 87.8 | 91.1 | 78.6 | 0.47 | |
| LLM Judge (Haiku)Cost=1.00×2026.06 | 92.8 | 81.5 | 94.5 | 100 | 95.8 | 92.1 | 84.9 | 1 | |
| Indicator ProbesCost=∼ 02026.06 | 90.8 | 81.6 | 95 | 96.2 | 95.7 | 93.6 | 65.6 | 1.2 | |
| LLM Judge (Qwen3)Cost=0.09×2026.06 | 88.6 | 81 | 95.7 | 96.6 | 93.1 | 89 | 65.8 | 1 |