Guarded Agent Evaluation on ASB latest (IPI)
5.5ASRReAct-llamafirewall
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ReAct-llamafirewallAgent Backbone=GPT-4o, Guardrail Model=TS-Guard2026.01 | 5.5 | 46.63 | |
| ReAct-TS-FlowAgent Backbone=GPT-4o, Guardrail Model=TS-Guard2026.01 | 6.19 | 49.01 | |
| ReAct-llamafirewallAgent Backbone=Qwen2.5-14B-Instruct, Guardrail Model=TS-Guard2026.01 | 6.5 | 50 | |
| ReAct-TS-FlowAgent Backbone=Qwen2.5-14B-Instruct, Guardrail Model=TS-Guard2026.01 | 7 | 58.12 | |
| ReAct-TS-FlowAgent Backbone=GPT-4o, Guardrail Model=ShieldAgent-THU2026.01 | 16.25 | 44.8 | |
| ReAct-llamafirewallAgent Backbone=GPT-4o, Guardrail Model=GPT-4o-mini2026.01 | 19.25 | 45.37 | |
| ReAct-llamafirewallAgent Backbone=Qwen2.5-14B-Instruct, Guardrail Model=GPT-4o-mini2026.01 | 22.75 | 49.25 | |
| ReAct-TS-FlowAgent Backbone=Qwen2.5-14B-Instruct, Guardrail Model=ShieldAgent-THU2026.01 | 38.25 | 45.25 | |
| ReAct-TS-FlowAgent Backbone=GPT-4o, Guardrail Model=Safiron2026.01 | 65.5 | 47.5 | |
| ReAct-TS-FlowAgent Backbone=Qwen2.5-14B-Instruct, Guardrail Model=Safiron2026.01 | 69.75 | 49.37 | |
| ReAct-sandwich defenseAgent Backbone=GPT-4o, Guardrail Model=Sandwich2026.01 | 72.98 | 46.41 | |
| ReAct-sandwich defenseAgent Backbone=Qwen2.5-14B-Instruct, Guardrail Model=Sandwich2026.01 | 79 | 56.62 | |
| ReActAgent Backbone=GPT-4o, Guardrail Model=None2026.01 | 80 | 48 | |
| ReActAgent Backbone=Qwen2.5-14B-Instruct, Guardrail Model=None2026.01 | 86.5 | 52.25 |