Trace-level safety monitoring on TRACE
90.1ROC-AUCMeerkat
Evaluation Results
| Method | Links | |
|---|---|---|
| MeerkatTPC=50, Model=GPT-5.4m2026.04 | 90.1 | |
| MonitorTPC=50, Model=GPT-5.4m2026.04 | 90.1 | |
| MeerkatTPC=10, Model=Qwen3.52026.04 | 87.8 | |
| MonitorTPC=10, Model=Qwen3.52026.04 | 87.6 | |
| MeerkatTPC=25, Model=GLM-52026.04 | 87.3 | |
| MeerkatTPC=50, Model=Qwen3.52026.04 | 87.1 | |
| MonitorTPC=10, Model=GPT-5.4m2026.04 | 86.4 | |
| MeerkatTPC=10, Model=GPT-5.4m2026.04 | 85.3 | |
| MonitorTPC=25, Model=GLM-52026.04 | 85.3 | |
| MonitorTPC=25, Model=GPT-5.4m2026.04 | 84.8 | |
| MeerkatTPC=25, Model=GPT-5.4m2026.04 | 84.7 | |
| MonitorTPC=50, Model=Qwen3.52026.04 | 84.4 | |
| MonitorTPC=25, Model=Qwen3.52026.04 | 83.5 | |
| MeerkatTPC=10, Model=GLM-52026.04 | 83.3 | |
| MonitorTPC=100, Model=GPT-5.4m2026.04 | 82.9 | |
| MeerkatTPC=25, Model=Qwen3.52026.04 | 82.8 | |
| MeerkatTPC=100, Model=GPT-5.4m2026.04 | 82.8 | |
| MonitorTPC=10, Model=GLM-52026.04 | 82.7 | |
| MeerkatTPC=50, Model=GLM-52026.04 | 81.8 | |
| MonitorTPC=50, Model=GLM-52026.04 | 81.8 | |
| MonitorTPC=100, Model=Qwen3.52026.04 | 81.2 | |
| MeerkatTPC=100, Model=Qwen3.52026.04 | 80.9 | |
| MeerkatTPC=100, Model=GLM-52026.04 | 79.8 | |
| MonitorTPC=100, Model=GLM-52026.04 | 79.8 | |
| Naive AgentTPC=10, Model=Qwen3.52026.04 | 78.2 | |
| Naive AgentTPC=25, Model=Qwen3.52026.04 | 68.1 | |
| Naive AgentTPC=50, Model=Qwen3.52026.04 | 67.2 | |
| Naive AgentTPC=100, Model=Qwen3.52026.04 | 58.7 |