Safety Classification on OAI (test)
86.5F1 ScoreOracle
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| OracleCondition=Perfect Routing2025.02 | 86.5 | 80.2 | 93.9 | 85.14 | |
| EntRouting=Entropy-based, Small Model=Llama-Guard-3-1B, Large Model=Granite-Guardian-3-8B2025.02 | 74.3 | 66.1 | 84.9 | 25.93 | |
| TSRouting=Thresholding, Small Model=Llama-Guard-3-1B, Large Model=Granite-Guardian-3-8B2025.02 | 74.3 | 64.1 | 88.5 | 62.34 | |
| Granite-Guardian-3-8BModel=Large (Granite-Guardian-3-8B)2025.02 | 73.1 | 61.7 | 89.7 | 62.64 | |
| CCRouting=Calibration-based, Small Model=Llama-Guard-3-1B, Large Model=Granite-Guardian-3-8B2025.02 | 73 | 62.9 | 87.2 | 25.76 | |
| SafeRouteRouting=SafeRoute (Ours), Small Model=Llama-Guard-3-1B, Large Model=Granite-Guardian-3-8B2025.02 | 71.5 | 58.9 | 91 | 21.02 | |
| RandomRouting=Random, Small Model=Llama-Guard-3-1B, Large Model=Granite-Guardian-3-8B2025.02 | 71.2 | 61.8 | 84.7 | 33.87 | |
| BCRouting=Bayesian-based, Small Model=Llama-Guard-3-1B, Large Model=Granite-Guardian-3-8B2025.02 | 70.7 | 58.4 | 89.5 | 25.51 | |
| Llama-Guard-3-1BModel=Small (Llama-Guard-3-1B)2025.02 | 67.9 | 58.7 | 80.5 | 9.32 |