Case-level safety detection on TRACE
0.901ROC-AUCMonitor
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| MonitorTPC=50, Model=Qwen3.52026.04 | 0.901 | — | |
| MeerkatTPC=100, Model=Qwen3.52026.04 | 0.87 | 0.023 | |
| MeerkatTPC=50, Model=Qwen3.52026.04 | 0.86 | -0.041 | |
| MeerkatTPC=10, Model=Qwen3.52026.04 | 0.852 | 0.017 | |
| MonitorTPC=100, Model=Qwen3.52026.04 | 0.847 | — | |
| MonitorTPC=10, Model=GLM-52026.04 | 0.838 | — | |
| MonitorTPC=10, Model=Qwen3.52026.04 | 0.835 | — | |
| MonitorTPC=50, Model=GLM-52026.04 | 0.821 | — | |
| Naive AgentTPC=50, Model=Qwen3.52026.04 | 0.818 | — | |
| MeerkatTPC=25, Model=Qwen3.52026.04 | 0.798 | 0.008 | |
| MonitorTPC=25, Model=GLM-52026.04 | 0.797 | — | |
| MonitorTPC=25, Model=Qwen3.52026.04 | 0.79 | — | |
| MonitorTPC=10, Model=GPT-5.4m2026.04 | 0.782 | — | |
| MeerkatTPC=100, Model=GLM-52026.04 | 0.762 | 0 | |
| MonitorTPC=100, Model=GLM-52026.04 | 0.762 | — | |
| MeerkatTPC=10, Model=GLM-52026.04 | 0.755 | -0.083 | |
| Naive AgentTPC=25, Model=Qwen3.52026.04 | 0.742 | — | |
| MeerkatTPC=25, Model=GLM-52026.04 | 0.742 | -0.055 | |
| MeerkatTPC=10, Model=GPT-5.4m2026.04 | 0.73 | -0.051 | |
| Naive AgentTPC=10, Model=Qwen3.52026.04 | 0.729 | — | |
| MeerkatTPC=50, Model=GLM-52026.04 | 0.716 | -0.105 | |
| MonitorTPC=50, Model=GPT-5.4m2026.04 | 0.699 | — | |
| MonitorTPC=25, Model=GPT-5.4m2026.04 | 0.692 | — | |
| Naive AgentTPC=100, Model=Qwen3.52026.04 | 0.658 | — | |
| MeerkatTPC=50, Model=GPT-5.4m2026.04 | 0.633 | -0.066 | |
| MonitorTPC=100, Model=GPT-5.4m2026.04 | 0.606 | — | |
| MeerkatTPC=100, Model=GPT-5.4m2026.04 | 0.582 | -0.025 | |
| MeerkatTPC=25, Model=GPT-5.4m2026.04 | 0.57 | -0.122 |