Safety Classification on HarmBench (test)
100F1 ScoreTPC
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| TPCOrder=5th order, Backbone=gemma-3-27b-it, Layer=40, Training Set=WildGuardTrain2025.09 | 100 | — | 100 | — | 100 | — | |
| EE-MLPOrder=5th order, Backbone=gemma-3-27b-it, Layer=40, Training Set=WildGuardTrain2025.09 | 99.75 | — | 99.5 | — | 99.5 | — | |
| Linear probeBackbone=gemma-3-27b-it, Layer=40, Training Set=WildGuardTrain2025.09 | 99.5 | — | 99 | — | 99 | — | |
| OracleCondition=Perfect Routing2025.02 | 90.5 | 88.5 | 92.7 | 40.13 | — | — | |
| EntRouting=Entropy-based, Small Model=Llama-Guard-3-1B, Large Model=Granite-Guardian-3-8B2025.02 | 84.4 | 83.5 | 85.4 | 15.31 | — | — | |
| CCRouting=Calibration-based, Small Model=Llama-Guard-3-1B, Large Model=Granite-Guardian-3-8B2025.02 | 84.3 | 80.6 | 88.3 | 14.74 | — | — | |
| SafeRouteRouting=SafeRoute (Ours), Small Model=Llama-Guard-3-1B, Large Model=Granite-Guardian-3-8B2025.02 | 83.4 | 79.6 | 87.6 | 10.32 | — | — | |
| BCRouting=Bayesian-based, Small Model=Llama-Guard-3-1B, Large Model=Granite-Guardian-3-8B2025.02 | 83.3 | 76.6 | 91.2 | 13.65 | — | — | |
| Granite-Guardian-3-8BModel=Large (Granite-Guardian-3-8B)2025.02 | 82 | 84.8 | 79.5 | 34.47 | — | — | |
| TSRouting=Thresholding, Small Model=Llama-Guard-3-1B, Large Model=Granite-Guardian-3-8B2025.02 | 80.5 | 84.1 | 77.3 | 38.77 | — | — | |
| RandomRouting=Random, Small Model=Llama-Guard-3-1B, Large Model=Granite-Guardian-3-8B2025.02 | 79.7 | 78.2 | 81.2 | 19.69 | — | — | |
| Llama-Guard-3-1BModel=Small (Llama-Guard-3-1B)2025.02 | 78.5 | 72.6 | 85.4 | 5.6 | — | — |