# Hijacking detection on Tool-input hijacking detection dataset 20% (test) Source: [Hijacking detection on Tool-input hijacking detection dataset 20% (test)]() Collection index: [llms.txt]() Data updated: 2026-04-07T06:48:41.102Z Evaluation of Hijacking detection performance on the 20% test split of the Tool-input hijacking detection dataset, measured using AUC-ROC, AUPRC, and TPR at 5% FPR. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Hijacking detection | | Dataset (including any stated split) | Tool-input hijacking detection dataset 20% (test) | | Metric | AUC-ROC | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [AUC-ROC](): Higher is better; unit: PERCENT. - [AUPRC](): Higher is better; unit: PERCENT. - [TPR@FPR5%](): Higher is better; unit: PERCENT. ## Recorded results | Method | AUC-ROC | Result date | Report paper | | --- | --- | --- | --- | | Probing | 100 | Not recorded | 69d48f747d0172140ec7422c | | Cosine Similarity | 100 | Not recorded | 69d48f747d0172140ec7422c | | Probing | 100 | Not recorded | 69d48f747d0172140ec7422c | | Probing | 99.97 | Not recorded | 69d48f747d0172140ec7422c | | Cosine Similarity | 99.68 | Not recorded | 69d48f747d0172140ec7422c | | Probing | 99.63 | Not recorded | 69d48f747d0172140ec7422c | | Probing | 99.31 | Not recorded | 69d48f747d0172140ec7422c | | Probing | 99.2 | Not recorded | 69d48f747d0172140ec7422c | | Probing | 99.12 | Not recorded | 69d48f747d0172140ec7422c | | Probing | 98.3 | Not recorded | 69d48f747d0172140ec7422c | | Cosine Similarity | 88.66 | Not recorded | 69d48f747d0172140ec7422c | | Probing | 87.76 | Not recorded | 69d48f747d0172140ec7422c | | Cosine Similarity | 87.05 | Not recorded | 69d48f747d0172140ec7422c | | Cosine Similarity | 86.01 | Not recorded | 69d48f747d0172140ec7422c | | Cosine Similarity | 85.6 | Not recorded | 69d48f747d0172140ec7422c | | Cosine Similarity | 83.79 | Not recorded | 69d48f747d0172140ec7422c | | Cosine Similarity | 79.21 | Not recorded | 69d48f747d0172140ec7422c | | Cosine Similarity | 75.94 | Not recorded | 69d48f747d0172140ec7422c | ### Probing AUC-ROC: 100. Result ID: 69d4a8c847f34ce924bbe359. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-32B","Layer":"53"}. This record does not assert independent reproduction. ### Cosine Similarity AUC-ROC: 100. Result ID: 69d4a8c847f34ce924bbe366. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Gemma3-12B","Layer":"5"}. This record does not assert independent reproduction. ### Probing AUC-ROC: 100. Result ID: 69d4a8c847f34ce924bbe367. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Mistral","Layer":"2"}. This record does not assert independent reproduction. ### Probing AUC-ROC: 99.97. Result ID: 69d4a8c847f34ce924bbe357. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-14B","Layer":"31"}. This record does not assert independent reproduction. ### Cosine Similarity AUC-ROC: 99.68. Result ID: 69d4a8c847f34ce924bbe35c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-4B","Layer":"4"}. This record does not assert independent reproduction. ### Probing AUC-ROC: 99.63. Result ID: 69d4a8c847f34ce924bbe35d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-8B","Layer":"22"}. This record does not assert independent reproduction. ### Probing AUC-ROC: 99.31. Result ID: 69d4a8c847f34ce924bbe35f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-14B","Layer":"18"}. This record does not assert independent reproduction. ### Probing AUC-ROC: 99.2. Result ID: 69d4a8c847f34ce924bbe361. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA3-8B","Layer":"5"}. This record does not assert independent reproduction. ### Probing AUC-ROC: 99.12. Result ID: 69d4a8c847f34ce924bbe35b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-4B","Layer":"4"}. This record does not assert independent reproduction. ### Probing AUC-ROC: 98.3. Result ID: 69d4a8c847f34ce924bbe363. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GLM4-9B","Layer":"23"}. This record does not assert independent reproduction. ### Cosine Similarity AUC-ROC: 88.66. Result ID: 69d4a8c847f34ce924bbe35e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-8B","Layer":"22"}. This record does not assert independent reproduction. ### Probing AUC-ROC: 87.76. Result ID: 69d4a8c847f34ce924bbe365. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Gemma3-12B","Layer":"5"}. This record does not assert independent reproduction. ### Cosine Similarity AUC-ROC: 87.05. Result ID: 69d4a8c847f34ce924bbe358. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-14B","Layer":"31"}. This record does not assert independent reproduction. ### Cosine Similarity AUC-ROC: 86.01. Result ID: 69d4a8c847f34ce924bbe364. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GLM4-9B","Layer":"23"}. This record does not assert independent reproduction. ### Cosine Similarity AUC-ROC: 85.6. Result ID: 69d4a8c847f34ce924bbe368. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Mistral","Layer":"2"}. This record does not assert independent reproduction. ### Cosine Similarity AUC-ROC: 83.79. Result ID: 69d4a8c847f34ce924bbe35a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-32B","Layer":"53"}. This record does not assert independent reproduction. ### Cosine Similarity AUC-ROC: 79.21. Result ID: 69d4a8c847f34ce924bbe362. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA3-8B","Layer":"5"}. This record does not assert independent reproduction. ### Cosine Similarity AUC-ROC: 75.94. Result ID: 69d4a8c847f34ce924bbe360. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-14B","Layer":"18"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.