List your company

Unsafe Prompt Detection on ToxicChat (test)

0.815PrecisionOpenAI Moderation API
0.218040.373020.5280.68298Feb 21, 2024Apr 21, 2024Jun 21, 2024Aug 20, 2024Oct 20, 2024Dec 19, 2024Feb 18, 2025

Evaluation results

MethodLinks
OpenAI Moderation API2024.020.8150.1450.246—
GradSafe-ZeroBase model=Llama-2-7b-chat-hf, Detection threshold=0.25, Gap threshold=12024.020.7530.6670.707—
Llama GuardBase model=Llama-2 7b, Training=Finetuned on 10,000 prompts2024.020.7440.3960.517—
OracleCondition=Perfect Routing2025.020.650.8810.74894.17
Perspective API2024.020.6140.1480.238—
Azure API2024.020.5590.6340.594—
GPT-4Model version=gpt-4-1106-preview, Zero-shot prompting=true2024.020.4750.8310.604—
TSRouting=Thresholding, Small Model=Llama-Guard-3-1B, Large Model=Granite-Guardian-3-8B2025.020.440.8320.57683.49
Granite-Guardian-3-8BModel=Large (Granite-Guardian-3-8B)2025.020.4230.8590.56793.55
EntRouting=Entropy-based, Small Model=Llama-Guard-3-1B, Large Model=Granite-Guardian-3-8B2025.020.4180.7160.52843.52
SafeRouteRouting=SafeRoute (Ours), Small Model=Llama-Guard-3-1B, Large Model=Granite-Guardian-3-8B2025.020.3950.7810.52535.12
CCRouting=Calibration-based, Small Model=Llama-Guard-3-1B, Large Model=Granite-Guardian-3-8B2025.020.3810.7510.50645.42
RandomRouting=Random, Small Model=Llama-Guard-3-1B, Large Model=Granite-Guardian-3-8B2025.020.3380.7160.46152.7
BCRouting=Bayesian-based, Small Model=Llama-Guard-3-1B, Large Model=Granite-Guardian-3-8B2025.020.3330.790.46847.12
Llama-Guard-3-1BModel=Small (Llama-Guard-3-1B)2025.020.2420.570.34117.2
Llama-2Model version=Llama-2-7b-chat-hf, Zero-shot prompting=true2024.020.2410.8220.373—