Misalignment Monitoring Dataset (dev)
0.962AUROC (Deception)LLM Judge (Haiku)
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| LLM Judge (Haiku)Cost=1.00×2026.06 | 0.962 | 0.998 | 0.964 | 0.97 | 0.957 | 0.959 | 86.7 | 100 | |
| Indicator ProbesCost=∼ 02026.06 | 0.926 | 0.971 | 0.973 | 0.988 | 0.94 | 0.971 | 77.8 | 93 | |
| LLM Judge (Haiku, ind)Cost=1.00×2026.06 | 0.91 | 0.998 | 0.967 | 0.955 | 0.961 | 0.95 | 83.3 | 47 | |
| LLM Judge (Qwen3, ind)Cost=0.09×2026.06 | 0.89 | 0.879 | 0.895 | 0.959 | 0.765 | 0.902 | 77.8 | 73 | |
| Probes + Haiku 4.5Cost=0.06×2026.06 | 0.879 | 0.997 | 0.966 | 0.988 | 0.977 | 0.976 | 88.9 | 33 | |
| LLM Judge (Qwen3)Cost=0.09×2026.06 | 0.834 | 0.936 | 0.951 | 0.854 | 0.81 | 0.865 | 55.6 | 100 |