Prefix-risk ranking on tau^2-Bench (held-out)
71AUPRCPrefixGuard (Transformer)
Evaluation Results
| Method | Links | |
|---|---|---|
| PrefixGuard (Transformer)Input view=StepView, Head / scorer=Transformer2026.05 | 71 | |
| PrefixGuard (GRU)Input view=StepView, Head / scorer=GRU2026.05 | 69.6 | |
| PrefixGuard (FSM)Input view=StepView, Head / scorer=FSM2026.05 | 61.4 | |
| Transformer (Raw-text control)Input view=Raw text, Head / scorer=Transformer2026.05 | 59.7 | |
| GRU (Raw-text control)Input view=Raw text, Head / scorer=GRU2026.05 | 55.4 | |
| FSM (Raw-text control)Input view=Raw text, Head / scorer=FSM2026.05 | 46.6 | |
| DeepSeek-V4-ProInput view=Prompt, Head / scorer=DeepSeek-V4-Pro2026.05 | 39.6 | |
| PrefixGuard (DFA)Input view=StepView, Head / scorer=DFA2026.05 | 31.6 | |
| GPT-5.4-miniInput view=Prompt, Head / scorer=GPT-5.4-mini2026.05 | 30.2 | |
| PPM LSTMInput view=StepView activity, Head / scorer=PPM LSTM2026.05 | 23.1 | |
| DFA (Raw-text control)Input view=Raw text, Head / scorer=DFA2026.05 | 22.2 |