# Unsafe Prompt Detection on ToxicChat (test)

Source: [Unsafe Prompt Detection on ToxicChat (test)](<https://www.sota2.com/research/sota/unsafe-prompt-detection-on-toxicchat-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:58:29.018Z

Evaluation of model performance on detecting unsafe prompts using the test split of the ToxicChat dataset, typically measured using Precision, Recall, and F1-score.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Unsafe Prompt Detection |
| Dataset (including any stated split) | ToxicChat (test) |
| Metric | Precision |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 16 of 16 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Precision](<https://www.sota2.com/research/sota/unsafe-prompt-detection-on-toxicchat-test.md?metric=Precision>): Higher is better; unit: SCALAR.

- [Recall](<https://www.sota2.com/research/sota/unsafe-prompt-detection-on-toxicchat-test.md?metric=Recall>): Higher is better; unit: SCALAR.

- [F1 Score](<https://www.sota2.com/research/sota/unsafe-prompt-detection-on-toxicchat-test.md?metric=F1%20Score>): Higher is better; unit: SCALAR.

- [Latency](<https://www.sota2.com/research/sota/unsafe-prompt-detection-on-toxicchat-test.md?metric=Latency>): Lower is better; unit: TIME.

## Recorded results

| Method | Precision | Result date | Report paper |
| --- | --- | --- | --- |
| OpenAI Moderation API | 0.815 | Not recorded | 68fa736c1e209131cea5c676 |
| GradSafe-Zero | 0.753 | Not recorded | 68fa736c1e209131cea5c676 |
| Llama Guard | 0.744 | Not recorded | 68fa736c1e209131cea5c676 |
| Oracle | 0.65 | Not recorded | 68fa78bc1e209131cea7a210 |
| Perspective API | 0.614 | Not recorded | 68fa736c1e209131cea5c676 |
| Azure API | 0.559 | Not recorded | 68fa736c1e209131cea5c676 |
| GPT-4 | 0.475 | Not recorded | 68fa736c1e209131cea5c676 |
| TS | 0.44 | Not recorded | 68fa78bc1e209131cea7a210 |
| Granite-Guardian-3-8B | 0.423 | Not recorded | 68fa78bc1e209131cea7a210 |
| Ent | 0.418 | Not recorded | 68fa78bc1e209131cea7a210 |
| SafeRoute | 0.395 | Not recorded | 68fa78bc1e209131cea7a210 |
| CC | 0.381 | Not recorded | 68fa78bc1e209131cea7a210 |
| Random | 0.338 | Not recorded | 68fa78bc1e209131cea7a210 |
| BC | 0.333 | Not recorded | 68fa78bc1e209131cea7a210 |
| Llama-Guard-3-1B | 0.242 | Not recorded | 68fa78bc1e209131cea7a210 |
| Llama-2 | 0.241 | Not recorded | 68fa736c1e209131cea5c676 |

### OpenAI Moderation API

Precision: 0.815. Result ID: 6992ea2a3a6c314849125606.

[Reporting paper](<https://www.sota2.com/research/paper/68fa736c1e209131cea5c676.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e8b1e209131cea40729.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### GradSafe-Zero

Precision: 0.753. Result ID: 6992ea2a3a6c31484912560c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa736c1e209131cea5c676.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa736c1e209131cea5c676.md>)

[Implementation](<https://github.com/xyq7/GradSafe>)

Recorded evaluation configuration: {"Base model":"Llama-2-7b-chat-hf","Detection threshold":0.25,"Gap threshold":1}. This record does not assert independent reproduction.

### Llama Guard

Precision: 0.744. Result ID: 6992ea2a3a6c31484912560b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa736c1e209131cea5c676.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa717b1e209131cea5186d.md>)

Recorded evaluation configuration: {"Base model":"Llama-2 7b","Training":"Finetuned on 10,000 prompts"}. This record does not assert independent reproduction.

### Oracle

Precision: 0.65. Result ID: 69959e423badfaffe69a671f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78bc1e209131cea7a210.md>)

Recorded evaluation configuration: {"Condition":"Perfect Routing"}. This record does not assert independent reproduction.

### Perspective API

Precision: 0.614. Result ID: 6992ea2a3a6c314849125607.

[Reporting paper](<https://www.sota2.com/research/paper/68fa736c1e209131cea5c676.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### Azure API

Precision: 0.559. Result ID: 6992ea2a3a6c314849125608.

[Reporting paper](<https://www.sota2.com/research/paper/68fa736c1e209131cea5c676.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### GPT-4

Precision: 0.475. Result ID: 6992ea2a3a6c314849125609.

[Reporting paper](<https://www.sota2.com/research/paper/68fa736c1e209131cea5c676.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa727a1e209131cea56a1e.md>)

[Implementation](<https://github.com/openai/evals>)

Recorded evaluation configuration: {"Model version":"gpt-4-1106-preview","Zero-shot prompting":true}. This record does not assert independent reproduction.

### TS

Precision: 0.44. Result ID: 69959e423badfaffe69a671b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78bc1e209131cea7a210.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa638b1e209131ce9ff6d1.md>)

Recorded evaluation configuration: {"Routing":"Thresholding","Small Model":"Llama-Guard-3-1B","Large Model":"Granite-Guardian-3-8B"}. This record does not assert independent reproduction.

### Granite-Guardian-3-8B

Precision: 0.423. Result ID: 69959e423badfaffe69a6718.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78bc1e209131cea7a210.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76331e209131cea6bcd3.md>)

Recorded evaluation configuration: {"Model":"Large (Granite-Guardian-3-8B)"}. This record does not assert independent reproduction.

### Ent

Precision: 0.418. Result ID: 69959e423badfaffe69a671a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78bc1e209131cea7a210.md>)

Recorded evaluation configuration: {"Routing":"Entropy-based","Small Model":"Llama-Guard-3-1B","Large Model":"Granite-Guardian-3-8B"}. This record does not assert independent reproduction.

### SafeRoute

Precision: 0.395. Result ID: 69959e423badfaffe69a671e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78bc1e209131cea7a210.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78bc1e209131cea7a210.md>)

Recorded evaluation configuration: {"Routing":"SafeRoute (Ours)","Small Model":"Llama-Guard-3-1B","Large Model":"Granite-Guardian-3-8B"}. This record does not assert independent reproduction.

### CC

Precision: 0.381. Result ID: 69959e423badfaffe69a671c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78bc1e209131cea7a210.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6a201e209131cea261f0.md>)

Recorded evaluation configuration: {"Routing":"Calibration-based","Small Model":"Llama-Guard-3-1B","Large Model":"Granite-Guardian-3-8B"}. This record does not assert independent reproduction.

### Random

Precision: 0.338. Result ID: 69959e423badfaffe69a6719.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78bc1e209131cea7a210.md>)

Recorded evaluation configuration: {"Routing":"Random","Small Model":"Llama-Guard-3-1B","Large Model":"Granite-Guardian-3-8B"}. This record does not assert independent reproduction.

### BC

Precision: 0.333. Result ID: 69959e423badfaffe69a671d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78bc1e209131cea7a210.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75f51e209131cea6aa55.md>)

Recorded evaluation configuration: {"Routing":"Bayesian-based","Small Model":"Llama-Guard-3-1B","Large Model":"Granite-Guardian-3-8B"}. This record does not assert independent reproduction.

### Llama-Guard-3-1B

Precision: 0.242. Result ID: 69959e423badfaffe69a6717.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78bc1e209131cea7a210.md>)

Recorded evaluation configuration: {"Model":"Small (Llama-Guard-3-1B)"}. This record does not assert independent reproduction.

### Llama-2

Precision: 0.241. Result ID: 6992ea2a3a6c31484912560a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa736c1e209131cea5c676.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa701f1e209131cea492fe.md>)

[Implementation](<https://github.com/facebookresearch/llama>)

Recorded evaluation configuration: {"Model version":"Llama-2-7b-chat-hf","Zero-shot prompting":true}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
