Misalignment Detection on Agent-SafetyBench underspecified queries n=148
0.8Error Rate (ER)ProvePrompt
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ProvePromptModel=Gemini 3 Flash2026.05 | 0.8 | — | — | |
| ProvenanceGuardModel=Gemini 3 Flash2026.05 | 1.1 | — | — | |
| ProvenanceGuardModel=Claude Sonnet 4.62026.05 | 1.1 | — | — | |
| ProvenanceGuardModel=GPT-5 Mini2026.05 | 1.2 | — | — | |
| ProvenanceGuardModel=GPT-4.12026.05 | 1.4 | — | — | |
| ProvenanceGuardModel=GPT-4.1 Mini2026.05 | 1.4 | — | — | |
| ProvenanceGuardModel=Gemini 3.1 Flash-Lite2026.05 | 1.4 | — | — | |
| ProvenanceGuardModel=Gemini 2.5 Flash2026.05 | 1.8 | — | — | |
| ProvenanceGuardModel=Claude Haiku 4.52026.05 | 2 | — | — | |
| ProvePromptModel=Claude Sonnet 4.62026.05 | 2.2 | — | — | |
| ProvenanceGuardModel=GPT-52026.05 | 2.6 | — | — | |
| ProvenanceGuardModel=Gemini 2.5 Flash-Lite2026.05 | 4.6 | — | — | |
| ProvePromptModel=Claude Haiku 4.52026.05 | 5 | — | — | |
| ProvePromptModel=GPT-4.12026.05 | 6.8 | — | — | |
| ProvePromptModel=Gemini 2.5 Flash2026.05 | 6.8 | — | — | |
| ProvePromptModel=Gemini 3.1 Flash-Lite2026.05 | 8.5 | — | — | |
| ProvePromptModel=GPT-52026.05 | 11.5 | — | — | |
| ProvePromptModel=GPT-5 Mini2026.05 | 12.3 | — | — | |
| ProvePromptModel=GPT-4.1 Mini2026.05 | 17.8 | — | — | |
| DirectPromptModel=Gemini 3 Flash2026.05 | 17.8 | — | — | |
| DirectPromptModel=GPT-52026.05 | 27.3 | — | — | |
| DirectPromptModel=Claude Haiku 4.52026.05 | 34.6 | — | — | |
| DirectPromptModel=GPT-5 Mini2026.05 | 38 | — | — | |
| DirectPromptModel=Claude Sonnet 4.62026.05 | 40 | — | — | |
| DirectPromptModel=GPT-4.12026.05 | 45.9 | — | — | |
| DirectPromptModel=Gemini 3.1 Flash-Lite2026.05 | 46.4 | — | — | |
| DirectPromptModel=Gemini 2.5 Flash2026.05 | 47 | — | — | |
| ProvePromptModel=Gemini 2.5 Flash-Lite2026.05 | 56.4 | — | — | |
| DirectPromptModel=GPT-4.1 Mini2026.05 | 60.8 | — | — | |
| DirectPromptModel=Gemini 2.5 Flash-Lite2026.05 | 70.9 | — | — |