Refusal Detection on WILDGUARD (test)
94F1 (Harmful)WILDGUARD
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| WILDGUARD2024.06 | 94 | 88.5 | 88.6 | 88.6 | |
| GPT-42024.06 | 93.9 | 91.4 | 93.5 | 92.4 | |
| HarmB-Llamabackbone=Llama2024.06 | 89.8 | 74 | 72.5 | 73.1 | |
| MD-Judge2024.06 | 85.7 | 50.6 | 59.4 | 55.5 | |
| LibrAI-LongFormer-refbackbone=LongFormer2024.06 | 84.1 | 71.2 | 69.3 | 70.1 | |
| Llama-Guard22024.06 | 82.2 | 47.9 | 58.8 | 53.8 | |
| BeaverDam2024.06 | 80.7 | 48.4 | 59.4 | 54.1 | |
| HarmB-Mistralbackbone=Mistral2024.06 | 79.3 | 56.1 | 60.5 | 58.6 | |
| LibrAI-LongFormer-harmbackbone=LongFormer2024.06 | 79.2 | 61.7 | 62.7 | 62.3 | |
| Llama-Guard2024.06 | 76.5 | 45.1 | 56.9 | 51.4 | |
| OAI Mod. API2024.06 | 71.4 | 44.5 | 54.3 | 49.8 | |
| Keyword-based2024.06 | 67.8 | 67 | 65.9 | 66.3 | |
| Aegis-Guard-Ptype=Permissive2024.06 | 67.1 | 45.4 | 48.2 | 46.9 | |
| Aegis-Guard-Dtype=Defensive2024.06 | 56.1 | 38.9 | 44 | 41.8 |