Safety Classification on ToxicChat (out-of-distribution)
72.88F1 ScoreMulti-head self-attn
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Multi-head self-attnTraining dataset=WildGuardMix, Backbone=Llama-3.2-3B, Added Params (M)=35, Extra LM call=false2026.01 | 72.88 | 0.798 | |
| WildGuardAdded Params (M)=7000, Extra LM call=true2026.01 | 70.8 | — | |
| Aegis-Guard-DAdded Params (M)=8000, Extra LM call=true2026.01 | 70 | — | |
| GPT-4Added Params (M)=1700000, Extra LM call=true2026.01 | 68.3 | — | |
| ShieldHead (Gemma2-27B)Added Params (M)=306, Extra LM call=false2026.01 | 67.7 | — | |
| Scoring attentionTraining dataset=WildGuardMix, Backbone=Llama-3.2-3B, Added Params (M)=0.1, Extra LM call=false2026.01 | 64.81 | 0.706 | |
| ShieldHead (Llama3.1-8B)Added Params (M)=91, Extra LM call=false2026.01 | 64.3 | — | |
| Llama GuardAdded Params (M)=7000, Extra LM call=true2026.01 | 61.6 | 0.626 | |
| OpenAI ModerationExtra LM call=true2026.01 | 61.4 | 0.631 | |
| Direct poolingTraining dataset=WildGuardMix, Backbone=Llama-3.2-3B, Added Params (M)=0.003, Extra LM call=false2026.01 | 53.33 | 0.565 | |
| Llama-Guard2Added Params (M)=8000, Extra LM call=true2026.01 | 47.1 | — |