Safety Guardrailing on ULSPB 350 interaction runs
0.02HS RateStateGuard (Base-Ensemble)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| StateGuard (Base-Ensemble)Backbone Model=GPT-5.4, Defense Setting=Base, Auditor Mode=Ensemble2026.05 | 0.02 | 58 | 10 | |
| StateGuard (Targeted-Ensemble)Backbone Model=Grok 4.20, Defense Setting=Targeted, Auditor Mode=Ensemble2026.05 | 0.03 | 61 | 4 | |
| StateGuard (Targeted-Ensemble)Backbone Model=GPT-5.4, Defense Setting=Targeted, Auditor Mode=Ensemble2026.05 | 0.05 | 59 | 4 | |
| StateGuard (Targeted-Ensemble)Backbone Model=Kimi K2.5, Defense Setting=Targeted, Auditor Mode=Ensemble2026.05 | 0.06 | 59 | 5 | |
| StateGuard (Base-Ensemble)Backbone Model=Grok 4.20, Defense Setting=Base, Auditor Mode=Ensemble2026.05 | 0.1 | 61 | 13 | |
| StateGuard (Base-Single)Backbone Model=GPT-5.4, Defense Setting=Base, Auditor Mode=Single-Auditor2026.05 | 0.11 | 57 | 14 | |
| StateGuard (Targeted-Single)Backbone Model=Grok 4.20, Defense Setting=Targeted, Auditor Mode=Single-Auditor2026.05 | 0.11 | 58 | 8 | |
| StateGuard (Base-Ensemble)Backbone Model=Kimi K2.5, Defense Setting=Base, Auditor Mode=Ensemble2026.05 | 0.12 | 56 | 12 | |
| StateGuard (Targeted-Ensemble)Backbone Model=MiniMax M2.7, Defense Setting=Targeted, Auditor Mode=Ensemble2026.05 | 0.14 | 49 | 5 | |
| StateGuard (Targeted-Single)Backbone Model=Kimi K2.5, Defense Setting=Targeted, Auditor Mode=Single-Auditor2026.05 | 0.23 | 57 | 8 | |
| StateGuard (Targeted-Single)Backbone Model=GPT-5.4, Defense Setting=Targeted, Auditor Mode=Single-Auditor2026.05 | 0.23 | 59 | 7 | |
| StateGuard (Base-Single)Backbone Model=Grok 4.20, Defense Setting=Base, Auditor Mode=Single-Auditor2026.05 | 0.27 | 60 | 17 | |
| StateGuard (Base-Single)Backbone Model=Kimi K2.5, Defense Setting=Base, Auditor Mode=Single-Auditor2026.05 | 0.36 | 56 | 16 | |
| StateGuard (Targeted-Single)Backbone Model=MiniMax M2.7, Defense Setting=Targeted, Auditor Mode=Single-Auditor2026.05 | 0.37 | 48 | 7 | |
| StateGuard (Base-Ensemble)Backbone Model=MiniMax M2.7, Defense Setting=Base, Auditor Mode=Ensemble2026.05 | 0.47 | 52 | 11 | |
| StateGuard (Base-Single)Backbone Model=MiniMax M2.7, Defense Setting=Base, Auditor Mode=Single-Auditor2026.05 | 0.95 | 54 | 15 | |
| ASB-PPLBackbone Model=Grok 4.202026.05 | 4.04 | 6 | 93 | |
| ASB-PPLBackbone Model=Kimi K2.52026.05 | 4.25 | 6 | 94 | |
| No DefenseBackbone Model=Grok 4.202026.05 | 4.35 | — | — | |
| No DefenseBackbone Model=Kimi K2.52026.05 | 4.42 | — | — | |
| ASB-PPLBackbone Model=GPT-5.42026.05 | 5.17 | 9 | 90 | |
| No DefenseBackbone Model=GPT-5.42026.05 | 5.53 | — | — | |
| ASB-PPLBackbone Model=MiniMax M2.72026.05 | 6.33 | 1 | 96 | |
| No DefenseBackbone Model=MiniMax M2.72026.05 | 6.42 | — | — |