# Adversarial Attack Detection on Gandalf

Source: [Adversarial Attack Detection on Gandalf](<https://www.sota2.com/research/sota/adversarial-attack-detection-on-gandalf>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T06:01:14.266Z

Evaluation of a model's ability to detect adversarial attacks on the Gandalf dataset, measured using metrics such as Recall.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Adversarial Attack Detection |
| Dataset (including any stated split) | Gandalf |
| Metric | Recall |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 15 of 15 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Recall](<https://www.sota2.com/research/sota/adversarial-attack-detection-on-gandalf.md?metric=Recall>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Recall | Result date | Report paper |
| --- | --- | --- | --- |
| Llama Prompt Guard 2 | 1 | Not recorded | 694b737a53fdd5c3617399ba |
| Apriel Guard | 0.91 | Not recorded | 694b737a53fdd5c3617399ba |
| Apriel Guard | 0.91 | Not recorded | 694b737a53fdd5c3617399ba |
| IBM Granite Guardian | 0.7 | Not recorded | 694b737a53fdd5c3617399ba |
| Qwen3Guard | 0.69 | Not recorded | 694b737a53fdd5c3617399ba |
| gpt-oss-safeguard | 0.63 | Not recorded | 694b737a53fdd5c3617399ba |
| IBM Granite Guardian | 0.52 | Not recorded | 694b737a53fdd5c3617399ba |
| IBM Granite Guardian | 0.47 | Not recorded | 694b737a53fdd5c3617399ba |
| IBM Granite Guardian | 0.44 | Not recorded | 694b737a53fdd5c3617399ba |
| IBM Granite Guardian | 0.41 | Not recorded | 694b737a53fdd5c3617399ba |
| Llama Guard | 0.27 | Not recorded | 694b737a53fdd5c3617399ba |
| Llama Guard | 0.26 | Not recorded | 694b737a53fdd5c3617399ba |
| Llama Guard | 0.23 | Not recorded | 694b737a53fdd5c3617399ba |
| Qwen3Guard | 0.02 | Not recorded | 694b737a53fdd5c3617399ba |
| ShieldGemma | 0 | Not recorded | 694b737a53fdd5c3617399ba |

### Llama Prompt Guard 2

Recall: 1. Result ID: 6996b3765698665ec4fdd652.

[Reporting paper](<https://www.sota2.com/research/paper/694b737a53fdd5c3617399ba.md>)

Recorded evaluation configuration: {"Model Size":"0.086B","Reasoning":false}. This record does not assert independent reproduction.

### Apriel Guard

Recall: 0.91. Result ID: 6996b3765698665ec4fdd659.

[Reporting paper](<https://www.sota2.com/research/paper/694b737a53fdd5c3617399ba.md>)

[Method origin paper](<https://www.sota2.com/research/paper/694b737a53fdd5c3617399ba.md>)

Recorded evaluation configuration: {"Model Size":"8B","Reasoning":false}. This record does not assert independent reproduction.

### Apriel Guard

Recall: 0.91. Result ID: 6996b3765698665ec4fdd65c.

[Reporting paper](<https://www.sota2.com/research/paper/694b737a53fdd5c3617399ba.md>)

[Method origin paper](<https://www.sota2.com/research/paper/694b737a53fdd5c3617399ba.md>)

Recorded evaluation configuration: {"Model Size":"8B","Reasoning":true}. This record does not assert independent reproduction.

### IBM Granite Guardian

Recall: 0.7. Result ID: 6996b3765698665ec4fdd654.

[Reporting paper](<https://www.sota2.com/research/paper/694b737a53fdd5c3617399ba.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76331e209131cea6bcd3.md>)

Recorded evaluation configuration: {"Version":"3.2","Model Size":"3B","Reasoning":false}. This record does not assert independent reproduction.

### Qwen3Guard

Recall: 0.69. Result ID: 6996b3765698665ec4fdd658.

[Reporting paper](<https://www.sota2.com/research/paper/694b737a53fdd5c3617399ba.md>)

Recorded evaluation configuration: {"Model Size":"8B","Constraint":"strict","Reasoning":false}. This record does not assert independent reproduction.

### gpt-oss-safeguard

Recall: 0.63. Result ID: 6996b3765698665ec4fdd65b.

[Reporting paper](<https://www.sota2.com/research/paper/694b737a53fdd5c3617399ba.md>)

Recorded evaluation configuration: {"Model Size":"20B","Reasoning":true}. This record does not assert independent reproduction.

### IBM Granite Guardian

Recall: 0.52. Result ID: 6996b3765698665ec4fdd653.

[Reporting paper](<https://www.sota2.com/research/paper/694b737a53fdd5c3617399ba.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76331e209131cea6bcd3.md>)

Recorded evaluation configuration: {"Version":"3.1","Model Size":"2B","Reasoning":false}. This record does not assert independent reproduction.

### IBM Granite Guardian

Recall: 0.47. Result ID: 6996b3765698665ec4fdd656.

[Reporting paper](<https://www.sota2.com/research/paper/694b737a53fdd5c3617399ba.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76331e209131cea6bcd3.md>)

Recorded evaluation configuration: {"Version":"3.3","Model Size":"8B","Reasoning":false}. This record does not assert independent reproduction.

### IBM Granite Guardian

Recall: 0.44. Result ID: 6996b3765698665ec4fdd65a.

[Reporting paper](<https://www.sota2.com/research/paper/694b737a53fdd5c3617399ba.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76331e209131cea6bcd3.md>)

Recorded evaluation configuration: {"Version":"3.3","Model Size":"8B","Reasoning":true}. This record does not assert independent reproduction.

### IBM Granite Guardian

Recall: 0.41. Result ID: 6996b3765698665ec4fdd655.

[Reporting paper](<https://www.sota2.com/research/paper/694b737a53fdd5c3617399ba.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76331e209131cea6bcd3.md>)

Recorded evaluation configuration: {"Version":"3.2","Model Size":"5B","Reasoning":false}. This record does not assert independent reproduction.

### Llama Guard

Recall: 0.27. Result ID: 6996b3765698665ec4fdd650.

[Reporting paper](<https://www.sota2.com/research/paper/694b737a53fdd5c3617399ba.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa717b1e209131cea5186d.md>)

Recorded evaluation configuration: {"Version":"3","Reasoning":false}. This record does not assert independent reproduction.

### Llama Guard

Recall: 0.26. Result ID: 6996b3765698665ec4fdd64f.

[Reporting paper](<https://www.sota2.com/research/paper/694b737a53fdd5c3617399ba.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa717b1e209131cea5186d.md>)

Recorded evaluation configuration: {"Version":"2","Reasoning":false}. This record does not assert independent reproduction.

### Llama Guard

Recall: 0.23. Result ID: 6996b3765698665ec4fdd651.

[Reporting paper](<https://www.sota2.com/research/paper/694b737a53fdd5c3617399ba.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa717b1e209131cea5186d.md>)

Recorded evaluation configuration: {"Version":"4","Reasoning":false}. This record does not assert independent reproduction.

### Qwen3Guard

Recall: 0.02. Result ID: 6996b3765698665ec4fdd657.

[Reporting paper](<https://www.sota2.com/research/paper/694b737a53fdd5c3617399ba.md>)

Recorded evaluation configuration: {"Model Size":"8B","Constraint":"loose","Reasoning":false}. This record does not assert independent reproduction.

### ShieldGemma

Recall: 0. Result ID: 6996b3765698665ec4fdd64e.

[Reporting paper](<https://www.sota2.com/research/paper/694b737a53fdd5c3617399ba.md>)

Recorded evaluation configuration: {"Model Size":"9B","Reasoning":false}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
