Safety Classification on WildGuardMix (test)
95.3F1 ScoreOSS-Safeguard-20B-High
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| OSS-Safeguard-20B-High2026.05 | 95.3 | — | — | — | — | 96.3 | 7.6 | |
| CSRMseverity augmentation=true, configurable safety configuration augmentation=false2026.05 | 95.3 | — | — | — | — | 98.5 | 3.3 | |
| CSRMseverity augmentation=true, configurable safety configuration augmentation=true2026.05 | 95.3 | — | — | — | — | 95.1 | 4.7 | |
| ShieldGemma-9B2026.05 | 95.1 | — | — | — | — | 97.9 | 4.2 | |
| LlamaGuard-3-8B2026.05 | 95 | — | — | — | — | 98.3 | 5.1 | |
| Llama-3.1-70B-InstChain-of-Thought=true2026.05 | 94 | — | — | — | — | 96.8 | 12.1 | |
| CSRMseverity augmentation=false, configurable safety configuration augmentation=false2026.05 | 93.6 | — | — | — | — | 97.9 | 5.7 | |
| Qwen3-30BThinking Mode=true2026.05 | 93.5 | — | — | — | — | 96.3 | 11 | |
| Llama-3.1-8B-Inst2026.05 | 91.2 | — | — | — | — | 96.8 | 11 | |
| WildGuardAdded Params (M)=7,000, Extra LM call=true, Serving protocol=extra model call2026.01 | 88.9 | — | — | — | — | — | — | |
| TPC (5th order)Backbone LLM=gemma-3-27B-it, Layer=322025.09 | 88.86 | — | — | — | — | — | — | |
| Bilinear probeBackbone LLM=gemma-3-27B-it, Layer=322025.09 | 88.79 | — | — | — | — | — | — | |
| Multi-head self-attnAdded Params (M)=35, Extra LM call=false, Serving protocol=same serving pass2026.01 | 88.55 | — | — | — | — | — | — | |
| MLPBackbone LLM=gemma-3-27B-it, Layer=322025.09 | 88.49 | — | — | — | — | — | — | |
| EE-MLP (5th exit)Backbone LLM=gemma-3-27B-it, Layer=322025.09 | 88.39 | — | — | — | — | — | — | |
| TPC (5th order)Backbone LLM=gpt-oss-20b, Layer=162025.09 | 88.05 | — | — | — | — | — | — | |
| Linear probeBackbone LLM=gemma-3-27B-it, Layer=322025.09 | 88.03 | — | — | — | — | — | — | |
| MLPBackbone LLM=gpt-oss-20b, Layer=162025.09 | 87.86 | — | — | — | — | — | — | |
| EE-MLP (5th exit)Backbone LLM=gpt-oss-20b, Layer=162025.09 | 87.31 | — | — | — | — | — | — | |
| Bilinear probeBackbone LLM=gpt-oss-20b, Layer=162025.09 | 87.13 | — | — | — | — | — | — | |
| Linear probeBackbone LLM=gpt-oss-20b, Layer=162025.09 | 86.7 | — | — | — | — | — | — | |
| Scoring attentionAdded Params (M)=0.10, Extra LM call=false, Serving protocol=same serving pass2026.01 | 85.98 | — | — | — | — | — | — | |
| TPC (5th order)Backbone LLM=Qwen3-30b-A3B-Base, Layer=322025.09 | 85.57 | — | — | — | — | — | — | |
| Linear probeBackbone LLM=Qwen3-30b-A3B-Base, Layer=322025.09 | 85.53 | — | — | — | — | — | — | |
| OracleCondition=Perfect Routing2025.02 | 85.5 | 91.2 | 80.5 | 104.22 | — | — | — | |
| MLPBackbone LLM=Qwen3-30b-A3B-Base, Layer=322025.09 | 85.48 | — | — | — | — | — | — | |
| EE-MLP (5th exit)Backbone LLM=Qwen3-30b-A3B-Base, Layer=322025.09 | 85.24 | — | — | — | — | — | — | |
| Bilinear probeBackbone LLM=Qwen3-30b-A3B-Base, Layer=322025.09 | 84.87 | — | — | — | — | — | — | |
| Bilinear probeBackbone LLM=Llama-3.2-3B, Layer=162025.09 | 84.78 | — | — | — | — | — | — | |
| TPC (5th order)Backbone LLM=Llama-3.2-3B, Layer=162025.09 | 84.48 | — | — | — | — | — | — | |
| EE-MLP (5th exit)Backbone LLM=Llama-3.2-3B, Layer=162025.09 | 83.84 | — | — | — | — | — | — | |
| MULI (logits, Llama-3.2-3B)Added Params (M)=0.13, Extra LM call=false, Serving protocol=same serving pass2026.01 | 83.79 | — | — | — | — | — | — | |
| MLPBackbone LLM=Llama-3.2-3B, Layer=162025.09 | 83.77 | — | — | — | — | — | — | |
| Linear probeBackbone LLM=Llama-3.2-3B, Layer=162025.09 | 83.24 | — | — | — | — | — | — | |
| Direct poolingAdded Params (M)=0.003, Extra LM call=false, Serving protocol=same serving pass2026.01 | 82.84 | — | — | — | — | — | — | |
| Llama3.1-AegisGuardAdded Params (M)=8,000, Extra LM call=true, Serving protocol=extra model call2026.01 | 82.1 | — | — | — | — | — | — | |
| DynaGuard-8BChain-of-Thought=true2026.05 | 79.3 | — | — | — | — | — | — | |
| SafeRouteRouting=SafeRoute (Ours), Small Model=Llama-Guard-3-1B, Large Model=Granite-Guardian-3-8B2025.02 | 75.3 | 82.5 | 69.3 | 25.07 | — | — | — | |
| Granite-Guardian-3-8BModel=Large (Granite-Guardian-3-8B)2025.02 | 75.1 | 77 | 73.4 | 87.94 | — | — | — | |
| EntRouting=Entropy-based, Small Model=Llama-Guard-3-1B, Large Model=Granite-Guardian-3-8B2025.02 | 75.1 | 79.5 | 71.3 | 37.93 | — | — | — | |
| TSRouting=Thresholding, Small Model=Llama-Guard-3-1B, Large Model=Granite-Guardian-3-8B2025.02 | 74.9 | 78.8 | 71.3 | 97.48 | — | — | — | |
| CCRouting=Calibration-based, Small Model=Llama-Guard-3-1B, Large Model=Granite-Guardian-3-8B2025.02 | 74.7 | 76.1 | 73.4 | 40.84 | — | — | — | |
| BCRouting=Bayesian-based, Small Model=Llama-Guard-3-1B, Large Model=Granite-Guardian-3-8B2025.02 | 74.2 | 71.6 | 76.9 | 46.9 | — | — | — | |
| RandomRouting=Random, Small Model=Llama-Guard-3-1B, Large Model=Granite-Guardian-3-8B2025.02 | 71.2 | 74.3 | 68.2 | 53.06 | — | — | — | |
| Llama-Guard2Added Params (M)=8,000, Extra LM call=true, Serving protocol=extra model call2026.01 | 70.4 | — | — | — | — | — | — | |
| Llama-Guard-3-1BModel=Small (Llama-Guard-3-1B)2025.02 | 66.4 | 67.7 | 65.3 | 14.9 | — | — | — | |
| OpenAI Mod APIExtra LM call=true, Serving protocol=extra model call2026.01 | 12.1 | — | — | — | — | — | — | |
| LEG baseTrain Dataset=AEGIS 2.0, Model Size=86M2026.01 | — | — | — | — | 60.4 | — | — | |
| LEG baseTrain Dataset=Wild-GuardMix, Model Size=86M2026.01 | — | — | — | — | 73.16 | — | — | |
| LEG baseTrain Dataset=Toxic-Chat0124, Model Size=86M2026.01 | — | — | — | — | 33.77 | — | — | |
| LEG largeTrain Dataset=AEGIS 2.0, Model Size=304M2026.01 | — | — | — | — | 66.66 | — | — | |
| LEG largeTrain Dataset=Wild-GuardMix, Model Size=304M2026.01 | — | — | — | — | 75.83 | — | — | |
| LEG largeTrain Dataset=Toxic-Chat0124, Model Size=304M2026.01 | — | — | — | — | 38.07 | — | — | |
| LEG xsTrain Dataset=AEGIS 2.0, Model Size=22M2026.01 | — | — | — | — | 53.28 | — | — | |
| LEG xsTrain Dataset=Wild-GuardMix, Model Size=22M2026.01 | — | — | — | — | 71.17 | — | — | |
| LEG xsTrain Dataset=Toxic-Chat0124, Model Size=22M2026.01 | — | — | — | — | 23.39 | — | — | |
| LIME Baseline baseTrain Dataset=AEGIS 2.0, Model Size=-2026.01 | — | — | — | — | 21.6 | — | — | |
| LIME Baseline baseTrain Dataset=Wild-GuardMix, Model Size=-2026.01 | — | — | — | — | 30.73 | — | — | |
| LIME Baseline baseTrain Dataset=Toxic-Chat0124, Model Size=-2026.01 | — | — | — | — | 20.11 | — | — | |
| LIME Baseline largeTrain Dataset=AEGIS 2.0, Model Size=-2026.01 | — | — | — | — | 22.89 | — | — | |
| LIME Baseline largeTrain Dataset=Wild-GuardMix, Model Size=-2026.01 | — | — | — | — | 33.4 | — | — | |
| LIME Baseline largeTrain Dataset=Toxic-Chat0124, Model Size=-2026.01 | — | — | — | — | 18.19 | — | — | |
| SHAP Baseline baseTrain Dataset=AEGIS 2.0, Model Size=-2026.01 | — | — | — | — | 25.73 | — | — | |
| SHAP Baseline baseTrain Dataset=Wild-GuardMix, Model Size=-2026.01 | — | — | — | — | 33.44 | — | — | |
| SHAP Baseline baseTrain Dataset=Toxic-Chat0124, Model Size=-2026.01 | — | — | — | — | 25.46 | — | — | |
| SHAP Baseline largeTrain Dataset=AEGIS 2.0, Model Size=-2026.01 | — | — | — | — | 32.16 | — | — | |
| SHAP Baseline largeTrain Dataset=Wild-GuardMix, Model Size=-2026.01 | — | — | — | — | 39.88 | — | — | |
| SHAP Baseline largeTrain Dataset=Toxic-Chat0124, Model Size=-2026.01 | — | — | — | — | 30.56 | — | — | |
| Word Baseline baseTrain Dataset=AEGIS 2.0, Model Size=86M2026.01 | — | — | — | — | 58.33 | — | — | |
| Word Baseline baseTrain Dataset=Wild-GuardMix, Model Size=86M2026.01 | — | — | — | — | 67.24 | — | — | |
| Word Baseline baseTrain Dataset=Toxic-Chat0124, Model Size=86M2026.01 | — | — | — | — | 46.32 | — | — | |
| Word Baseline largeTrain Dataset=AEGIS 2.0, Model Size=304M2026.01 | — | — | — | — | 61.98 | — | — | |
| Word Baseline largeTrain Dataset=Wild-GuardMix, Model Size=304M2026.01 | — | — | — | — | 70.36 | — | — | |
| Word Baseline largeTrain Dataset=Toxic-Chat0124, Model Size=304M2026.01 | — | — | — | — | 47.89 | — | — |