# Defense Robustness on Direct Inquiry Vanilla Source: [Defense Robustness on Direct Inquiry Vanilla]() Collection index: [llms.txt]() Data updated: 2026-03-24T12:44:40.923Z Evaluation of model defense robustness using the Direct Inquiry Vanilla dataset, measuring performance based on keyword matching, answer quality, and overall usefulness. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Defense Robustness | | Dataset (including any stated split) | Direct Inquiry Vanilla | | Metric | Keyword Match Rate | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 16 of 16 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Keyword Match Rate](): Higher is better; unit: PERCENT. - [Answer Quality Score](): Higher is better; unit: PERCENT. - [Overall Usefulness Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Keyword Match Rate | Result date | Report paper | | --- | --- | --- | --- | | STL | 100 | Not recorded | 69c21a840636e5bf4afc47aa | | DPL | 100 | Not recorded | 69c21a840636e5bf4afc47aa | | ES2 | 100 | Not recorded | 69c21a840636e5bf4afc47aa | | ES2 | 100 | Not recorded | 69c21a840636e5bf4afc47aa | | ES2 | 100 | Not recorded | 69c21a840636e5bf4afc47aa | | Base Model | 99 | Not recorded | 69c21a840636e5bf4afc47aa | | STL | 98 | Not recorded | 69c21a840636e5bf4afc47aa | | DPL | 97 | Not recorded | 69c21a840636e5bf4afc47aa | | DPL | 97 | Not recorded | 69c21a840636e5bf4afc47aa | | Base Model | 96 | Not recorded | 69c21a840636e5bf4afc47aa | | STL | 95 | Not recorded | 69c21a840636e5bf4afc47aa | | Base Model | 92 | Not recorded | 69c21a840636e5bf4afc47aa | | ES2 | 67 | Not recorded | 69c21a840636e5bf4afc47aa | | DPL | 47 | Not recorded | 69c21a840636e5bf4afc47aa | | STL | 42 | Not recorded | 69c21a840636e5bf4afc47aa | | Base Model | 30 | Not recorded | 69c21a840636e5bf4afc47aa | ### STL Keyword Match Rate: 100. Result ID: 69c2864147f34ce924aa881a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"\*Llama-2 (7B-Chat)"}. This record does not assert independent reproduction. ### DPL Keyword Match Rate: 100. Result ID: 69c2864147f34ce924aa881b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"\*Llama-2 (7B-Chat)"}. This record does not assert independent reproduction. ### ES2 Keyword Match Rate: 100. Result ID: 69c2864147f34ce924aa881c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"\*Llama-2 (7B-Chat)"}. This record does not assert independent reproduction. ### ES2 Keyword Match Rate: 100. Result ID: 69c2864147f34ce924aa8820. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Llama-3 (8B-Instruct)"}. This record does not assert independent reproduction. ### ES2 Keyword Match Rate: 100. Result ID: 69c2864147f34ce924aa8828. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Qwen-2.5 (7B-Instruct)"}. This record does not assert independent reproduction. ### Base Model Keyword Match Rate: 99. Result ID: 69c2864147f34ce924aa8819. [Reporting paper]() Recorded evaluation configuration: {"LLM":"\*Llama-2 (7B-Chat)"}. This record does not assert independent reproduction. ### STL Keyword Match Rate: 98. Result ID: 69c2864147f34ce924aa881e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Llama-3 (8B-Instruct)"}. This record does not assert independent reproduction. ### DPL Keyword Match Rate: 97. Result ID: 69c2864147f34ce924aa881f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Llama-3 (8B-Instruct)"}. This record does not assert independent reproduction. ### DPL Keyword Match Rate: 97. Result ID: 69c2864147f34ce924aa8827. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Qwen-2.5 (7B-Instruct)"}. This record does not assert independent reproduction. ### Base Model Keyword Match Rate: 96. Result ID: 69c2864147f34ce924aa881d. [Reporting paper]() Recorded evaluation configuration: {"LLM":"Llama-3 (8B-Instruct)"}. This record does not assert independent reproduction. ### STL Keyword Match Rate: 95. Result ID: 69c2864147f34ce924aa8826. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Qwen-2.5 (7B-Instruct)"}. This record does not assert independent reproduction. ### Base Model Keyword Match Rate: 92. Result ID: 69c2864147f34ce924aa8825. [Reporting paper]() Recorded evaluation configuration: {"LLM":"Qwen-2.5 (7B-Instruct)"}. This record does not assert independent reproduction. ### ES2 Keyword Match Rate: 67. Result ID: 69c2864147f34ce924aa8824. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Mistral (7B-Instruct)"}. This record does not assert independent reproduction. ### DPL Keyword Match Rate: 47. Result ID: 69c2864147f34ce924aa8823. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Mistral (7B-Instruct)"}. This record does not assert independent reproduction. ### STL Keyword Match Rate: 42. Result ID: 69c2864147f34ce924aa8822. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Mistral (7B-Instruct)"}. This record does not assert independent reproduction. ### Base Model Keyword Match Rate: 30. Result ID: 69c2864147f34ce924aa8821. [Reporting paper]() Recorded evaluation configuration: {"LLM":"Mistral (7B-Instruct)"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.