# Misalignment Detection on Agent-SafetyBench underspecified queries n=148 Source: [Misalignment Detection on Agent-SafetyBench underspecified queries n=148]() Collection index: [llms.txt]() Data updated: 2026-07-03T06:22:16.309Z Evaluation of model performance on detecting misalignment using the specific subset of 148 underspecified queries from the Agent-SafetyBench dataset, measured by Error Rate (ER), Delta, and p-value. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Misalignment Detection | | Dataset (including any stated split) | Agent-SafetyBench underspecified queries n=148 | | Metric | Error Rate (ER) | | Unit | PERCENT | | Direction | Lower is better | | Results loaded | 30 of 30 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Error Rate (ER)](): Lower is better; unit: PERCENT. - [Delta](): Lower is better; unit: SCALAR. - [p-value](): Lower is better; unit: SCALAR. ## Recorded results | Method | Error Rate (ER) | Result date | Report paper | | --- | --- | --- | --- | | ProvePrompt | 0.8 | Not recorded | 6a4741ee143ebdcacacc5a79 | | ProvenanceGuard | 1.1 | Not recorded | 6a4741ee143ebdcacacc5a79 | | ProvenanceGuard | 1.1 | Not recorded | 6a4741ee143ebdcacacc5a79 | | ProvenanceGuard | 1.2 | Not recorded | 6a4741ee143ebdcacacc5a79 | | ProvenanceGuard | 1.4 | Not recorded | 6a4741ee143ebdcacacc5a79 | | ProvenanceGuard | 1.4 | Not recorded | 6a4741ee143ebdcacacc5a79 | | ProvenanceGuard | 1.4 | Not recorded | 6a4741ee143ebdcacacc5a79 | | ProvenanceGuard | 1.8 | Not recorded | 6a4741ee143ebdcacacc5a79 | | ProvenanceGuard | 2 | Not recorded | 6a4741ee143ebdcacacc5a79 | | ProvePrompt | 2.2 | Not recorded | 6a4741ee143ebdcacacc5a79 | | ProvenanceGuard | 2.6 | Not recorded | 6a4741ee143ebdcacacc5a79 | | ProvenanceGuard | 4.6 | Not recorded | 6a4741ee143ebdcacacc5a79 | | ProvePrompt | 5 | Not recorded | 6a4741ee143ebdcacacc5a79 | | ProvePrompt | 6.8 | Not recorded | 6a4741ee143ebdcacacc5a79 | | ProvePrompt | 6.8 | Not recorded | 6a4741ee143ebdcacacc5a79 | | ProvePrompt | 8.5 | Not recorded | 6a4741ee143ebdcacacc5a79 | | ProvePrompt | 11.5 | Not recorded | 6a4741ee143ebdcacacc5a79 | | ProvePrompt | 12.3 | Not recorded | 6a4741ee143ebdcacacc5a79 | | ProvePrompt | 17.8 | Not recorded | 6a4741ee143ebdcacacc5a79 | | DirectPrompt | 17.8 | Not recorded | 6a4741ee143ebdcacacc5a79 | | DirectPrompt | 27.3 | Not recorded | 6a4741ee143ebdcacacc5a79 | | DirectPrompt | 34.6 | Not recorded | 6a4741ee143ebdcacacc5a79 | | DirectPrompt | 38 | Not recorded | 6a4741ee143ebdcacacc5a79 | | DirectPrompt | 40 | Not recorded | 6a4741ee143ebdcacacc5a79 | | DirectPrompt | 45.9 | Not recorded | 6a4741ee143ebdcacacc5a79 | | DirectPrompt | 46.4 | Not recorded | 6a4741ee143ebdcacacc5a79 | | DirectPrompt | 47 | Not recorded | 6a4741ee143ebdcacacc5a79 | | ProvePrompt | 56.4 | Not recorded | 6a4741ee143ebdcacacc5a79 | | DirectPrompt | 60.8 | Not recorded | 6a4741ee143ebdcacacc5a79 | | DirectPrompt | 70.9 | Not recorded | 6a4741ee143ebdcacacc5a79 | ### ProvePrompt Error Rate (ER): 0.8. Result ID: 6a4754fa60b88bb33991b264. [Reporting paper]() Recorded evaluation configuration: {"Model":"Gemini 3 Flash"}. This record does not assert independent reproduction. ### ProvenanceGuard Error Rate (ER): 1.1. Result ID: 6a4754fa60b88bb33991b263. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Gemini 3 Flash"}. This record does not assert independent reproduction. ### ProvenanceGuard Error Rate (ER): 1.1. Result ID: 6a4754fa60b88bb33991b26f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Claude Sonnet 4.6"}. This record does not assert independent reproduction. ### ProvenanceGuard Error Rate (ER): 1.2. Result ID: 6a4754fa60b88bb33991b25a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-5 Mini"}. This record does not assert independent reproduction. ### ProvenanceGuard Error Rate (ER): 1.4. Result ID: 6a4754fa60b88bb33991b25d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-4.1"}. This record does not assert independent reproduction. ### ProvenanceGuard Error Rate (ER): 1.4. Result ID: 6a4754fa60b88bb33991b260. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-4.1 Mini"}. This record does not assert independent reproduction. ### ProvenanceGuard Error Rate (ER): 1.4. Result ID: 6a4754fa60b88bb33991b266. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Gemini 3.1 Flash-Lite"}. This record does not assert independent reproduction. ### ProvenanceGuard Error Rate (ER): 1.8. Result ID: 6a4754fa60b88bb33991b269. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Gemini 2.5 Flash"}. This record does not assert independent reproduction. ### ProvenanceGuard Error Rate (ER): 2. Result ID: 6a4754fa60b88bb33991b272. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Claude Haiku 4.5"}. This record does not assert independent reproduction. ### ProvePrompt Error Rate (ER): 2.2. Result ID: 6a4754fa60b88bb33991b270. [Reporting paper]() Recorded evaluation configuration: {"Model":"Claude Sonnet 4.6"}. This record does not assert independent reproduction. ### ProvenanceGuard Error Rate (ER): 2.6. Result ID: 6a4754fa60b88bb33991b257. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-5"}. This record does not assert independent reproduction. ### ProvenanceGuard Error Rate (ER): 4.6. Result ID: 6a4754fa60b88bb33991b26c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Gemini 2.5 Flash-Lite"}. This record does not assert independent reproduction. ### ProvePrompt Error Rate (ER): 5. Result ID: 6a4754fa60b88bb33991b273. [Reporting paper]() Recorded evaluation configuration: {"Model":"Claude Haiku 4.5"}. This record does not assert independent reproduction. ### ProvePrompt Error Rate (ER): 6.8. Result ID: 6a4754fa60b88bb33991b25e. [Reporting paper]() Recorded evaluation configuration: {"Model":"GPT-4.1"}. This record does not assert independent reproduction. ### ProvePrompt Error Rate (ER): 6.8. Result ID: 6a4754fa60b88bb33991b26a. [Reporting paper]() Recorded evaluation configuration: {"Model":"Gemini 2.5 Flash"}. This record does not assert independent reproduction. ### ProvePrompt Error Rate (ER): 8.5. Result ID: 6a4754fa60b88bb33991b267. [Reporting paper]() Recorded evaluation configuration: {"Model":"Gemini 3.1 Flash-Lite"}. This record does not assert independent reproduction. ### ProvePrompt Error Rate (ER): 11.5. Result ID: 6a4754fa60b88bb33991b258. [Reporting paper]() Recorded evaluation configuration: {"Model":"GPT-5"}. This record does not assert independent reproduction. ### ProvePrompt Error Rate (ER): 12.3. Result ID: 6a4754fa60b88bb33991b25b. [Reporting paper]() Recorded evaluation configuration: {"Model":"GPT-5 Mini"}. This record does not assert independent reproduction. ### ProvePrompt Error Rate (ER): 17.8. Result ID: 6a4754fa60b88bb33991b261. [Reporting paper]() Recorded evaluation configuration: {"Model":"GPT-4.1 Mini"}. This record does not assert independent reproduction. ### DirectPrompt Error Rate (ER): 17.8. Result ID: 6a4754fa60b88bb33991b265. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Gemini 3 Flash"}. This record does not assert independent reproduction. ### DirectPrompt Error Rate (ER): 27.3. Result ID: 6a4754fa60b88bb33991b259. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"GPT-5"}. This record does not assert independent reproduction. ### DirectPrompt Error Rate (ER): 34.6. Result ID: 6a4754fa60b88bb33991b274. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Claude Haiku 4.5"}. This record does not assert independent reproduction. ### DirectPrompt Error Rate (ER): 38. Result ID: 6a4754fa60b88bb33991b25c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"GPT-5 Mini"}. This record does not assert independent reproduction. ### DirectPrompt Error Rate (ER): 40. Result ID: 6a4754fa60b88bb33991b271. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Claude Sonnet 4.6"}. This record does not assert independent reproduction. ### DirectPrompt Error Rate (ER): 45.9. Result ID: 6a4754fa60b88bb33991b25f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"GPT-4.1"}. This record does not assert independent reproduction. ### DirectPrompt Error Rate (ER): 46.4. Result ID: 6a4754fa60b88bb33991b268. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Gemini 3.1 Flash-Lite"}. This record does not assert independent reproduction. ### DirectPrompt Error Rate (ER): 47. Result ID: 6a4754fa60b88bb33991b26b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Gemini 2.5 Flash"}. This record does not assert independent reproduction. ### ProvePrompt Error Rate (ER): 56.4. Result ID: 6a4754fa60b88bb33991b26d. [Reporting paper]() Recorded evaluation configuration: {"Model":"Gemini 2.5 Flash-Lite"}. This record does not assert independent reproduction. ### DirectPrompt Error Rate (ER): 60.8. Result ID: 6a4754fa60b88bb33991b262. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"GPT-4.1 Mini"}. This record does not assert independent reproduction. ### DirectPrompt Error Rate (ER): 70.9. Result ID: 6a4754fa60b88bb33991b26e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Gemini 2.5 Flash-Lite"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.