Task-specific Risk Detection on EICU-AC (test)
98.7LPAGuardAgent
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| GuardAgentType=Guardrail-based, Backbone=GPT-42025.02 | 98.7 | 100 | 97.5 | 98.7 | 98.7 | |
| AGrailType=Guardrail-based, Backbone=Claude-3.52025.02 | 98.4 | 97 | 100 | 98.5 | 100 | |
| Claude-3.5Type=Model-based, Shots=0-shot2025.02 | 97.9 | 95.9 | 100 | 97.9 | 100 | |
| AGrailType=Guardrail-based, Backbone=GPT-4o2025.02 | 97.8 | 97.5 | 98.1 | 97.8 | 100 | |
| GPT-4oType=Model-based, Shots=0-shot2025.02 | 97.2 | 94.7 | 100 | 97.3 | 100 | |
| Claude-3.5Type=Model-based, Shots=1-shot2025.02 | 94.6 | 95.3 | 94.4 | 94.7 | 100 | |
| GPT-4oType=Model-based, Shots=1-shot2025.02 | 92.1 | 89.6 | 95.7 | 92.5 | 100 | |
| AgentMonitorType=Guardrail-based, Backbone=GPT-4o2025.02 | 82.3 | 98.2 | 66.7 | 79.4 | 100 | |
| LLaMA-Guard3Type=Guardrail-based2025.02 | 48.7 | — | 0 | — | — |