Misalignment Detection on Agent-SafetyBench n=35 (aligned traces)
31.4Avg. IRDirectPrompt
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| DirectPromptModel=Gemini 2.5 Flash-Lite2026.05 | 31.4 | — | — | |
| ProvePromptModel=Claude Haiku 4.52026.05 | 31.4 | 0.057 | 10.7 | |
| DirectPromptModel=Claude Haiku 4.52026.05 | 25.7 | — | — | |
| ProvenanceGuardModel=Gemini 2.5 Flash-Lite2026.05 | 18.9 | -0.126 | 91 | |
| ProvenanceGuardModel=GPT-4.1 Mini2026.05 | 17.7 | 0.08 | 16 | |
| ProvePromptModel=Gemini 2.5 Flash2026.05 | 17.7 | 0.091 | 1.3 | |
| ProvenanceGuardModel=Gemini 3 Flash2026.05 | 17.1 | 0.131 | 4.2 | |
| ProvenanceGuardModel=Gemini 3.1 Flash-Lite2026.05 | 17.1 | 0.114 | 6.3 | |
| ProvenanceGuardModel=Claude Haiku 4.52026.05 | 17.1 | -0.086 | 85 | |
| ProvenanceGuardModel=GPT-4.12026.05 | 13.7 | 0.074 | 13.8 | |
| ProvePromptModel=Gemini 2.5 Flash-Lite2026.05 | 13.7 | -0.177 | 100 | |
| ProvenanceGuardModel=GPT-5 Mini2026.05 | 10.9 | 0.08 | 10.6 | |
| ProvenanceGuardModel=Claude Sonnet 4.62026.05 | 10.9 | 0.08 | 12.3 | |
| DirectPromptModel=GPT-4.1 Mini2026.05 | 9.7 | — | — | |
| ProvePromptModel=Gemini 3 Flash2026.05 | 9.7 | 0.057 | 6.6 | |
| ProvenanceGuardModel=Gemini 2.5 Flash2026.05 | 9.1 | 0.006 | 48.3 | |
| ProvePromptModel=Gemini 3.1 Flash-Lite2026.05 | 8.6 | 0.029 | 30.7 | |
| DirectPromptModel=Gemini 2.5 Flash2026.05 | 8.6 | — | — | |
| ProvenanceGuardModel=GPT-52026.05 | 6.9 | 0.057 | 12.7 | |
| DirectPromptModel=GPT-4.12026.05 | 6.3 | — | — | |
| ProvePromptModel=Claude Sonnet 4.62026.05 | 6.3 | 0.034 | 7.7 | |
| ProvePromptModel=GPT-4.12026.05 | 5.7 | -0.006 | 52.7 | |
| DirectPromptModel=Gemini 3.1 Flash-Lite2026.05 | 5.7 | — | — | |
| ProvePromptModel=GPT-52026.05 | 4.6 | 0.034 | 12.1 | |
| ProvePromptModel=GPT-4.1 Mini2026.05 | 4.6 | -0.051 | 90.4 | |
| DirectPromptModel=Gemini 3 Flash2026.05 | 4 | — | — | |
| DirectPromptModel=GPT-5 Mini2026.05 | 2.9 | — | — | |
| DirectPromptModel=Claude Sonnet 4.62026.05 | 2.9 | — | — | |
| DirectPromptModel=GPT-52026.05 | 1.1 | — | — | |
| ProvePromptModel=GPT-5 Mini2026.05 | 1.1 | -0.017 | 74.3 |