Prefix-risk ranking on TerminalBench (held-out)
0.557AUPRCPrefixGuard (GRU)
Evaluation Results
| Method | Links | |
|---|---|---|
| PrefixGuard (GRU)Input view=StepView, Head / scorer=GRU2026.05 | 0.557 | |
| PrefixGuard (Transformer)Input view=StepView, Head / scorer=Transformer2026.05 | 0.555 | |
| PrefixGuard (FSM)Input view=StepView, Head / scorer=FSM2026.05 | 0.447 | |
| GRU (Raw-text control)Input view=Raw text, Head / scorer=GRU2026.05 | 0.37 | |
| Transformer (Raw-text control)Input view=Raw text, Head / scorer=Transformer2026.05 | 0.363 | |
| FSM (Raw-text control)Input view=Raw text, Head / scorer=FSM2026.05 | 0.272 | |
| PrefixGuard (DFA)Input view=StepView, Head / scorer=DFA2026.05 | 0.184 | |
| DFA (Raw-text control)Input view=Raw text, Head / scorer=DFA2026.05 | 0.137 | |
| GPT-5.4-miniInput view=Prompt, Head / scorer=GPT-5.4-mini2026.05 | 0.127 | |
| DeepSeek-V4-ProInput view=Prompt, Head / scorer=DeepSeek-V4-Pro2026.05 | 0.107 | |
| PPM LSTMInput view=StepView activity, Head / scorer=PPM LSTM2026.05 | 0.093 |