# Unsafe Prompt Detection on XSTest (test)

Source: [Unsafe Prompt Detection on XSTest (test)](<https://www.sota2.com/research/sota/unsafe-prompt-detection-on-xstest-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:58:26.925Z

Evaluation of model performance on detecting unsafe prompts using the test split of the XSTest dataset, reporting Precision, Recall, and F1-score.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Unsafe Prompt Detection |
| Dataset (including any stated split) | XSTest (test) |
| Metric | Precision |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 7 of 7 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Precision](<https://www.sota2.com/research/sota/unsafe-prompt-detection-on-xstest-test.md?metric=Precision>): Higher is better; unit: PERCENT.

- [Recall](<https://www.sota2.com/research/sota/unsafe-prompt-detection-on-xstest-test.md?metric=Recall>): Higher is better; unit: PERCENT.

- [F1-score](<https://www.sota2.com/research/sota/unsafe-prompt-detection-on-xstest-test.md?metric=F1-score>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Precision | Result date | Report paper |
| --- | --- | --- | --- |
| OpenAI Moderation API | 87.8 | Not recorded | 68fa736c1e209131cea5c676 |
| GPT-4 | 87.8 | Not recorded | 68fa736c1e209131cea5c676 |
| GradSafe-Zero | 85.6 | Not recorded | 68fa736c1e209131cea5c676 |
| Perspective API | 83.5 | Not recorded | 68fa736c1e209131cea5c676 |
| Llama Guard | 81.3 | Not recorded | 68fa736c1e209131cea5c676 |
| Azure API | 67.3 | Not recorded | 68fa736c1e209131cea5c676 |
| Llama-2 | 50.9 | Not recorded | 68fa736c1e209131cea5c676 |

### OpenAI Moderation API

Precision: 87.8. Result ID: 6992ea2a3a6c31484912560d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa736c1e209131cea5c676.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e8b1e209131cea40729.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### GPT-4

Precision: 87.8. Result ID: 6992ea2a3a6c314849125610.

[Reporting paper](<https://www.sota2.com/research/paper/68fa736c1e209131cea5c676.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa727a1e209131cea56a1e.md>)

[Implementation](<https://github.com/openai/evals>)

Recorded evaluation configuration: {"Model version":"gpt-4-1106-preview","Zero-shot prompting":true}. This record does not assert independent reproduction.

### GradSafe-Zero

Precision: 85.6. Result ID: 6992ea2a3a6c314849125613.

[Reporting paper](<https://www.sota2.com/research/paper/68fa736c1e209131cea5c676.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa736c1e209131cea5c676.md>)

[Implementation](<https://github.com/xyq7/GradSafe>)

Recorded evaluation configuration: {"Base model":"Llama-2-7b-chat-hf","Detection threshold":0.25,"Gap threshold":1}. This record does not assert independent reproduction.

### Perspective API

Precision: 83.5. Result ID: 6992ea2a3a6c31484912560e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa736c1e209131cea5c676.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### Llama Guard

Precision: 81.3. Result ID: 6992ea2a3a6c314849125612.

[Reporting paper](<https://www.sota2.com/research/paper/68fa736c1e209131cea5c676.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa717b1e209131cea5186d.md>)

Recorded evaluation configuration: {"Base model":"Llama-2 7b","Training":"Finetuned on 10,000 prompts"}. This record does not assert independent reproduction.

### Azure API

Precision: 67.3. Result ID: 6992ea2a3a6c31484912560f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa736c1e209131cea5c676.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### Llama-2

Precision: 50.9. Result ID: 6992ea2a3a6c314849125611.

[Reporting paper](<https://www.sota2.com/research/paper/68fa736c1e209131cea5c676.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa701f1e209131cea492fe.md>)

[Implementation](<https://github.com/facebookresearch/llama>)

Recorded evaluation configuration: {"Model version":"Llama-2-7b-chat-hf","Zero-shot prompting":true}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
