Safety Evaluation on HEx-PHI (Harmfulness Score, ASR, Win Rate)
2.06Harmfulness ScoreTOSS-Pro
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| TOSS-ProModel=Llama-2-7B-Chat-hf2026.03 | 2.06 | — | — | |
| TOSS-ProModel=Llama-3-8B-Instruct2026.03 | 2.09 | — | — | |
| TOSSModel=Llama-2-7B-Chat-hf2026.03 | 2.43 | — | — | |
| TOSSBackbone=Llama-3-8B-Instruct2026.03 | 2.48 | 77.18 | 90.8 | |
| RandomModel=Llama-2-7B-Chat-hf2026.03 | 2.63 | — | — | |
| SafeInstrModel=Llama-2-7B-Chat-hf2026.03 | 2.66 | — | — | |
| TOSSModel=Llama-3-8B-Instruct2026.03 | 2.72 | — | — | |
| Standard SFTModel=Llama-2-7B-Chat-hf2026.03 | 2.74 | — | — | |
| SEALModel=Llama-2-7B-Chat-hf2026.03 | 2.88 | — | — | |
| Standard SFTBackbone=Llama-3-8B-Instruct2026.03 | 3.01 | 94.68 | 50 | |
| DSIRModel=Llama-2-7B-Chat-hf2026.03 | 3.01 | — | — | |
| SEALModel=Llama-3-8B-Instruct2026.03 | 3.02 | — | — | |
| RandomModel=Llama-3-8B-Instruct2026.03 | 3.52 | — | — | |
| SafeInstrModel=Llama-3-8B-Instruct2026.03 | 4.04 | — | — | |
| Standard SFTModel=Llama-3-8B-Instruct2026.03 | 4.18 | — | — | |
| DSIRModel=Llama-3-8B-Instruct2026.03 | 4.37 | — | — | |
| Circuit BreakersParadigm=Training-based defense, Base Model=Llama-3-8B-Instruct2026.03 | — | — | 44 | |
| SafetyLockParadigm=Training-free defense, Base Model=Llama-3-8B-Instruct2026.03 | — | — | 50.34 | |
| SEALBackbone=Llama-3-8B-Instruct2026.03 | — | — | 53.04 | |
| Standard SFTParadigm=/, Base Model=Llama-3-8B-Instruct2026.03 | — | — | 50 | |
| Standard SFTBackbone=Llama-3-8B-Instruct2026.03 | — | — | 50 | |
| TOSSParadigm=Training-based defense, Base Model=Llama-3-8B-Instruct2026.03 | — | — | 87.54 | |
| TOSSBackbone=Llama-3-8B-Instruct2026.03 | — | — | 93.18 |