Policy Violation Detection on SB+SI
69True PositivesVIGIL
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| VIGILTests=1522026.06 | 69 | 8 | 89.6 | 95.8 | 92.6 | |
| LLM-as-judgeTests=1522026.06 | 48 | 5 | 90.6 | 66.7 | 76.8 | |
| ProgentTests=1522026.06 | 40 | 47 | 46 | 55.6 | 50.3 | |
| AgentSpecTests=1522026.06 | 34 | 12 | 73.9 | 47.2 | 57.6 |