Prefix-risk ranking on SkillsBench (held-out)
53.3AUPRCPrefixGuard (GRU)
Evaluation Results
| Method | Links | |
|---|---|---|
| PrefixGuard (GRU)Input view=StepView, Head / scorer=GRU2026.05 | 53.3 | |
| PrefixGuard (Transformer)Input view=StepView, Head / scorer=Transformer2026.05 | 47.8 | |
| Transformer (Raw-text control)Input view=Raw text, Head / scorer=Transformer2026.05 | 31.5 | |
| GRU (Raw-text control)Input view=Raw text, Head / scorer=GRU2026.05 | 31.5 | |
| PrefixGuard (FSM)Input view=StepView, Head / scorer=FSM2026.05 | 27.3 | |
| FSM (Raw-text control)Input view=Raw text, Head / scorer=FSM2026.05 | 26 | |
| PrefixGuard (DFA)Input view=StepView, Head / scorer=DFA2026.05 | 19 | |
| DFA (Raw-text control)Input view=Raw text, Head / scorer=DFA2026.05 | 14.7 | |
| GPT-5.4-miniInput view=Prompt, Head / scorer=GPT-5.4-mini2026.05 | 10.1 | |
| PPM LSTMInput view=StepView activity, Head / scorer=PPM LSTM2026.05 | 8.9 | |
| DeepSeek-V4-ProInput view=Prompt, Head / scorer=DeepSeek-V4-Pro2026.05 | 8 |